What Counts as a Lasting Employment Effect?
The U.S. Census hiring study and the Swedish Stockholm Jobs evaluation do not offer a simple before-and-after story. This chapter focuses on the harder question underneath their results: compared with what would have happened without a temporary public job, did people work more, earn more, or merely move through a different benefit system? The answer gives readers a way to distinguish a local employment effect from a broad average, a wage effect from a work-incidence effect, and a benefit transition from a new job.
That distinction matters because participants are rarely a random slice of unemployed people. A person selected for a municipal job may be more motivated, healthier, easier to place, or already closer to employment than someone not selected. Conversely, a caseworker may direct especially difficult cases into a program. If we compare the two groups without accounting for those differences, the program can look better or worse than it really is.
The missing path: what would have happened without the job
The missing path is the counterfactual: the outcome the same person would have experienced at the same time if they had not received the temporary job. We can never observe both paths for one individual. A credible evaluation therefore tries to construct a comparison group that resembles the treated group in the features that matter for the outcome, while using a source of variation that is not itself a result of those features.
The U.S. Census study has two such comparisons, answering different questions. Its strongest result comes from a hiring threshold. Non-supervisory applicants generally needed a converted test score of 70 to be considered for employment. Applicants just below and just above that cutoff were close in measured ability and, before Census hiring began, had similar labor-market outcomes. Crossing the threshold increased the probability of actually being hired by about 6 percentage points. The researchers then use that threshold-induced change in hiring to estimate the effect of Census employment on later outcomes.
This is called a fuzzy regression discontinuity because the cutoff changes the chance of treatment rather than assigning it perfectly. A score just above 70 did not guarantee a job: the Census process still involved a background check, an interview, availability, training, fingerprinting, and employment-eligibility verification. Nor did every person below 70 necessarily have every other opportunity removed. The design asks a narrower question: what was the effect of the extra Census hiring generated by barely crossing the score threshold?
That narrowness is a strength, not an embarrassment. People close to a cutoff are plausibly comparable in ways that broad groups are not. But the resulting estimate belongs to applicants at that margin. It should not be silently converted into the effect of giving any temporary public job to any unemployed person. The Census paper's marginal applicants had particularly weak outside options: those around the threshold had lower prior employment, lower prior W-2 income, and much lower college completion than high-scoring applicants. The estimate is therefore most informative about people near the edge of employment.
The study also checks whether the threshold comparison is being distorted. Applicants just above and below the cutoff look broadly similar on many observed characteristics, and there is a strong first-stage relationship between crossing the cutoff and hiring. The authors find no pre-2009 discontinuity in the labor-market outcomes used for the analysis. They examine retaking behavior because applicants might have tried to cross the threshold deliberately; just-above-threshold applicants were, if anything, slightly less likely to have retaken the test. A formal density test is difficult to interpret because the number of applicants rises sharply at low scores, so these checks improve confidence without making the design assumption disappear.
The Census researchers use a second design to study applicants beyond the very bottom of the score distribution. Each job opening had a selection sheet defined by the position, location, schedule, and transportation required. Among non-veterans with the same score and the same status, applicants were randomly ordered. At the lowest score that received offers for a particular opening, an applicant's place in that random order affected whether they were among the initial offers and therefore more likely to be interviewed and hired. The study uses this ordering, along with selection-sheet fixed effects, to compare people who were alike in score and opening but differed in hiring likelihood.
This second comparison is broader but answers a different question. It does not reproduce the score-70 margin. The modal lowest offered score was 97, and most selection sheets stopped at 90 or above. Its estimate therefore describes applicants who generally had stronger alternatives than the threshold group. The first-stage effect is about a 10-percentage-point increase in eventual hiring for people receiving the initial-offer position. Because the source of variation is different, the resulting employment estimate should not be averaged together with the threshold estimate as if both described the same population.
Stockholm Jobs presents a third version of the counterfactual problem. Entry was voluntary for people offered a job, but caseworkers largely determined referrals, and clients could start a program at different points in an unemployment spell. A simple comparison of everyone who participated with everyone who did not would be especially vulnerable to selection and timing. Someone who remains at a job center long enough to receive an offer is not necessarily comparable with someone who left quickly for work or another reason.
The Swedish authors address this with dynamic inverse-probability weighting. In plain terms, they estimate how likely a person was to enter each program at each point in their job-center spell, using observed information such as age, education, prior earnings, social-assistance and labor-market history, a proxy for motivation, and health indicators. They then give greater weight to never-treated people who had remained in the initial state for a comparable length of time. Controls also receive an imputed start date so that outcomes are measured on a similar clock.
The procedure is more than a technical adjustment. It recognizes that “never treated” can be a selective category: people with short unemployment spells will be overrepresented unless the comparison accounts for the time they were at risk of treatment. The authors report near-zero pre-program effects and similar estimates when adding covariates or changing assignment-period groupings. Those results support the comparison, but the key conditional-independence assumption cannot be tested. Caseworkers may observe motivation, health, or readiness in ways the administrative data do not capture. The Swedish estimates are consequently careful observational treatment effects on the treated, not randomized workplace assignments.
The two evaluation strategies answer a common question with different scopes. The U.S. threshold design gains credibility from local quasi-random variation but applies to a narrow group. The U.S. within-score design extends across more applicants but uses a different source of random ordering and still identifies a particular set of compliers. The Swedish design covers three real program types and follows administrative outcomes, but it depends on the assumption that the important selection factors were observed. Knowing those boundaries is part of knowing what the estimates mean.
The diagram shows why a participant's later status is not enough. The observed path can be compared with a constructed no-job path, but the comparison must also specify what is being measured. A later W-2, a wage conditional on having a W-2, and a change from social assistance to unemployment insurance are different claims.
Employment, wages, and benefits are different ledgers
The Census study makes the distinction between working and earning especially visible. Its main employment outcome is whether an applicant had a positive W-2 in a given year. That is an extensive-margin measure: did an employer report any wages at all? The study also examines the log value of W-2 wages conditional on a positive W-2, an intensive-margin measure, and total W-2 wages that include people with zero reported wages.
For marginal Census applicants, hiring produced a large increase in positive W-2 receipt: about 61 percentage points in 2010. Yet the researchers do not find a lasting statistically significant increase in log wages among those who were working. Census work itself could lower the annual wage figure in 2010 because many jobs were part-time or part-year. Afterward, people who worked earned broadly comparable wage amounts to those who were not hired. The persistent difference was primarily a difference in whether someone worked, not a premium paid to someone already working.
This is an important correction to the phrase “better labor-market outcomes.” If a program raises the probability of working from 40 percent to 65 percent but leaves the wage of workers unchanged, it has created a substantial change in annual income for the population while creating no conditional wage advantage. The total-W-2 measure captures that distinction. The U.S. authors estimate nearly $6,000 in higher annual W-2 wages after 2010 for marginal hires, arising from more people having reported work.
The study's fiscal calculation follows the same logic. Assuming a 10 percent tax rate, a 3 percent discount rate, an average annual increase of $5,855, and a twenty-year period, the authors calculate more than $8,600 in additional discounted tax revenue per marginal hire. That figure is not an observed return on investment. It is an illustration of how a repeated employment difference can accumulate. It does not include the value of Census services, the cost of recruiting and supervising workers, replacement costs, possible private-sector displacement, or all transfers and taxes.
The broader Census comparison reinforces the need to label the margin being discussed. For the within-score applicant pool, hiring raised positive W-2 employment by about 22 percentage points at the 2010 peak, and the effect dissipated by 2011. The effect was about 46 points for scores 70–80, 23 points for scores 82–90, and 16 points for scores 92–100 in 2010. The paper does not detect a persistent post-2010 employment effect for the non-marginal group. That is evidence that the incremental value of the job was smaller when people had stronger outside options; it is not evidence that the Census job harmed all higher-scoring workers or that no small effect existed.
Stockholm Jobs requires a different separation. There, employment is measured as positive monthly earnings, including subsidized employment. The evaluation also records months and amounts of social assistance, or SA, and unemployment insurance, or UI. These programs are not two names for the same outcome. SA is means-tested and locally administered. UI is connected to work and fund-membership rules and is paid from central funds. A participant can therefore stop receiving SA because they found a job, because they became eligible for UI, or because the household's circumstances changed.
The regular-workplace results show why several ledgers can improve together. Over the period from month 13 through month 36 after program start, Youth employment participants gained 2.72 months of employment and about SEK 37,651 in earnings relative to weighted controls. They received 2.6 fewer months of SA and 1.26 more months of UI. The paper's combined calculation gives them about SEK 26,300 more income over the two years after the temporary job ended. These are not identical measures: fewer SA months may reflect higher earnings, a new job, or eligibility for UI, while additional employment months provide the direct labor-market evidence.
Other municipal employments also took place at regular workplaces, but the participant group was older and had a worse prior labor-market history. Their estimated gain over the same post-program window was 1.76 employment months and SEK 21,312 in earnings. SA receipt fell by six months, while UI receipt rose by 7.81 months. The paper calculates an approximately SEK 8,600 net income increase after combining earnings and benefits. The larger UI movement relative to the employment gain is a reminder that a program can change income support more quickly than it changes attachment to ordinary employment.
Stockholm hosts show the danger of reading one benefit outcome as a labor-market result. Compared with weighted controls, hosts had 1.7 fewer employment months and SEK 39,100 less earnings in the post-program period. They also received 5.37 fewer months of SA and 8.99 more months of UI. Their employment effect reached about negative 14.2 percentage points in month 26, while the UI effect was as high as roughly 60 percentage points in months 16–19. A reduction in locally financed assistance occurred alongside worse employment and a shift into centrally financed UI.
The comparison therefore supports a disciplined classification of claims. “Participants received less social assistance” is a benefit claim. “Participants worked more months” is an employment claim. “Participants earned more” is an earnings claim. “The municipality saved money” is a fiscal-incidence claim that requires the financing rules and administrative costs. One result can support one of these claims without supporting the others.
A final caution concerns the word “lasting.” The U.S. study observes positive W-2 receipt through 2023 for marginal applicants, while the Swedish study follows participants for three years after program start and summarizes two years after the program ended. A persistent effect is a pattern over the available observation window, not proof that the effect will continue forever. Conversely, a failure to detect a persistent effect is not proof that every individual experienced no change. The relevant question is always: persistent for whom, measured how, against which counterfactual, and with what uncertainty?
Once those questions are in place, the headline findings become more useful. The U.S. Census job appears to have changed the work incidence of applicants near the employment margin. Stockholm's regular-workplace programs changed both employment and benefit paths for vulnerable participants, while Stockholm hosts changed the benefit path without producing the same employment bridge. The next task is to look closely at the Census hiring process itself, because the meaning of that temporary job may have begun before the first day of work—with the test, screening, and public credibility of being selected.