What the Experiment Shows—and What It Does Not

The central evidence about Overstapbuddy.nl is an access experiment: interested job seekers were randomly offered the chance to use a platform that could connect them with a successful career-switcher. This chapter separates what that design can identify from what it cannot. The result is strongest for later employment and earnings in the study population, especially among people unemployed for longer than 17 weeks. The claims about a completed buddy relationship, the exact mechanism, and wider applicability are more limited.

The causal question is an offer, not a mentorship

The study recruited job seekers in two waves. An initial information message went to 6,048 eligible people in the first wave and 4,972 in the second. Of those who expressed interest—367 and 346 respectively—half were invited to create an account and half were assigned to a control or waitlist condition. In total, 713 job seekers took part. The main analysis selected the 524 people who had worked zero hours in the month before the invitation: 264 treated participants and 260 controls.

This sequence matters because there are several different questions one might ask. Did the invitation improve outcomes? Did creating an account improve outcomes? Did forming a buddy relationship improve outcomes? Did a particular kind of conversation improve outcomes? The randomized comparison answers the first question most cleanly. It compares people who were offered access with similar interested people who were not offered access at that point. It does not randomly assign a completed conversation.

The technical name for the main estimate is the intention-to-treat effect. In plain language, it is the effect of being placed in the opportunity to use the platform, whether or not the person takes every step afterward. Among treated job seekers, 53% created an account and 19% found a buddy. The paper reports 49 matches overall. These rates are crucial for interpretation. If we compared the 49 matched pairs with everyone else, we would also be comparing people who chose to sign up, judged a profile suitable, sent a request, and received an acceptance with people who did not make those choices. Their underlying readiness and circumstances could differ.

Randomizing the invitation does not erase every limitation. The causal claim applies to the defined group: interested job seekers in the Dutch public-employment-service setting, with the study’s occupational, résumé, language, and benefit-eligibility conditions. It also includes any effect of the invitation itself, such as seeing the transition video or thinking differently about a buddy, not only the effect of a completed match. The researchers note that interpreting the result as an effect on people who actually received treatment would require an additional assumption that the invitation did not cause people to seek an equivalent buddy through their own networks.

This is not a semantic technicality. A service provider deciding whether to offer a buddy platform needs to know whether it is buying a reliable dose of mentoring or creating a chance for a minority of people to form a useful relationship.