The Hidden Cost of Missing Context

The METR evidence gives us a way to look past the 19% headline and ask what developers actually did with the time. This chapter follows the AI-assisted workflow from prompt to integrated change, then examines why repository familiarity, tacit knowledge, complexity, and output reliability can change the balance between help and overhead. The goal is to distinguish observed activity shifts from plausible explanations rather than treating every correlation as a proven cause.

The assistant does not replace the whole workflow

When AI was allowed in METR’s study, developers spent a smaller share of their time actively coding and reading or searching for information. They spent more time prompting AI systems, waiting for generations, reviewing outputs, and idling. That shift is important because a model’s output is not the same thing as a completed software change. The generated text must cross several human and technical boundaries before it becomes part of a repository.

METR’s screen recordings made those boundaries visible. Researchers manually labeled 143 hours of recordings, using labels detailed enough to distinguish writing code, testing, reading, prompting, waiting, reviewing, and other activities. After filtering recordings with problems such as substantial gaps or mismatches between the recording and reported time, a smaller valid set supported the detailed comparisons. This is unusually direct evidence about how time was allocated, although it is still a subset of the full work and may not represent every kind of issue.

The path is not linear. A developer begins with an issue and whatever repository knowledge they carry. They formulate a request, wait for a model response, inspect the response, and decide whether it is useful. If they accept it, they may clean up the code, run tests, read related files, and prepare the change for review. If they reject it, they may try another prompt, provide more context, write the change themselves, or revert an earlier attempt. Each branch can be productive in some circumstances. Each can also consume time without moving the issue closer to an acceptable result.

flowchart TD A["Issue plus human repository context"] --> B["Prompt AI"] B --> W["Wait for generation"] W --> C["Review generated output"] C --> D{"Useful enough to keep?"} D -->|Yes| E["Clean up and adapt"] D -->|No| R["Reject, revise prompt, or write directly"] R --> B E --> F["Run tests and prepare change"] F --> G{"Meets repository review bar?"} G -->|Yes| H["Integrated delivered change"] G -->|No| E

The recorded numbers put scale on this workflow. In the subset used for the time-allocation analysis, developers spent about 4% of their time waiting for AI generations and about 9% reviewing or cleaning generated output when AI was available. Those percentages are not enormous on their own, but they are nontrivial, especially when the output is only a starting point. Developers accepted fewer than 44% of generations. Among surveyed participants, 75% said they read every line of AI-generated code they used, 56% often made major changes to it, and all reported modifying generated code.

Those observations do not mean that every rejected generation was wasted. A rejected suggestion may reveal a useful approach, expose a missing requirement, or help a developer decide what not to do.