What Coding Agents Change: Work, Throughput, and Scale
Once engineers return to coding agents for recurring tasks, the important question shifts from adoption to consequence: what kind of work changes, and what can the resulting metrics actually prove? This chapter examines the enterprise evidence for merged-PR throughput alongside the public evidence about contribution size and task type. The aim is to distinguish a durable association with more output from the stronger claim that agents create more valuable software.
The distinction matters because an agent can change the work before any pull request is counted. It may make a previously deferred refactoring approachable, allow a developer to run several work streams in parallel, or make a larger feature cheap enough to attempt. The repository then records a change set, not the counterfactual task that was never attempted. A useful interpretation must therefore connect the task portfolio, the agent loop, human review, and the output metric without allowing any one of them to stand in for the others.
The throughput signal is real, but its meaning is conditional
The Microsoft study used two complementary designs to examine merged pull requests. The first aligned a cohort of engineers who adopted Claude Code or Copilot CLI during the rollout week with a control built from engineers who did not use either tool. It compared their daily merged-PR trajectories before and after the intervention and estimated a synthetic counterfactual for what adopter output might have looked like without adoption.
Over the post-period, the synthetic-control analysis estimated a 24.0% lift in merged PRs per engineer per day. The 95% credible interval ran from 14.5% to 33.7%, and the posterior tail-area p-value was below 0.001. This is more informative than simply comparing a month before with a month after: the model uses the pre-period trajectories of control series to represent a counterfactual trend. It nevertheless remains dependent on the quality of that counterfactual and on the assumption that adopter-specific shocks did not account for the entire difference.
The study also asked whether the lift looked like a launch effect that would disappear. It did not find statistically distinguishable decay over the roughly four-month observation window. The estimate was 29.4% in February and 20.0% from March 1 through April 29; the intervals overlapped and both excluded zero. The appropriate conclusion is not that coding agents guarantee a permanent productivity increase. It is that, within this setting and time horizon, the observed association did not fade in the way a short-lived novelty effect would predict.
The second design stayed within the adopter population and made each engineer their own comparison. It compared weeks in which a person used an agentic command-line tool with weeks in which that same person recorded no tool-use days, while also accounting for common calendar shocks.