How an Exposure Measure Becomes a Map
The OECD occupational exposure map is built from two judgements that must meet: what an occupation requires and what an AI system can do. This chapter focuses on the construction of the occupation side, from a deliberately diverse human benchmark to a repeated multi-model extension across hundreds of occupations. Understanding those handoffs helps a reader see both why the resulting map is scalable and why its scores remain estimates rather than occupational ground truth.
The apparent simplicity of an exposure table can conceal a long chain of choices. Someone must decide which tasks represent an occupation, how a capability scale applies to those tasks, whether a disagreement reflects an error or a real ambiguity, and how to extend a small set of carefully considered ratings to a much larger occupational space. The OECD’s method makes those choices visible enough to examine. That is a strength, but visibility is not the same as certainty.
Build a human benchmark before scaling
The OECD began with the practical problem that manually rating roughly 900 occupations would be too costly and inconsistent to do as one undifferentiated exercise. It therefore selected 40 occupations designed to span distinct capability profiles: reasoning-intensive work, social interaction, physical and perceptual demands, different knowledge areas, occupational groups, and job zones. The purpose was not to create a miniature ranking of prestigious and low-status jobs. It was to make sure the benchmark contained contrasting combinations of capabilities against which an expanded map could be tested.
That sampling decision is consequential. If the initial occupations all resembled office-based analytical roles, a later system might learn a narrow picture of what work requires. A benchmark that includes teachers, healthcare roles, managers, therapists, drivers, tradespeople, service workers, and other contrasting profiles creates more demanding reference points. It forces the mapping process to confront cases where language and reasoning coexist with physical work, where social interaction is central, or where a role looks routine but still involves judgement and adaptation.
The benchmark was then rated by OECD analysts using the nine capability indicators. They were given common descriptions of the occupations, selected tasks, and the level descriptions for the capability scales. The question was not whether an exceptional worker might display a capability, nor whether a future technology could reorganise the role. It was what level of capability the occupation, as currently performed, essentially requires. That distinction keeps the occupation-side rating tied to the present object being measured, even though the resulting exposure index is intended to be useful for thinking about future AI progress.
Individual ratings were only the first pass. The analysts compared cases of substantial disagreement in a Delphi-like consensus process and revised their judgements. This step did more than produce a cleaner average. It exposed where the framework itself was difficult to apply. A disagreement about a bus driver might arise because one rater focuses on driving while another includes cleaning and luggage handling. A disagreement about a healthcare occupation might reflect different assumptions about how much empathy or professional judgement is essential. Those are not interchangeable problems: one concerns which tasks define the occupation, while the other concerns the meaning of a capability level.
Consensus helps with disagreements that are matters of oversight or inconsistent application. If a rater missed a negotiating task in a jewellery occupation, collective review can correct the omission. If raters use the word “dialogue” mechanically and assign a high Language level to every job involving conversation, discussion can restore the intended distinction between occasional talk and substantive dialogue as a defining activity. In these cases, calibration improves the measurement.
But consensus cannot make every ambiguity disappear. The group found persistent difficulty distinguishing Manipulation from Robotic Intelligence. Some raters treated the latter as a broad form of physical intelligence integrating perception, movement, and environmental navigation; others interpreted it more narrowly. Similar overlap appeared between higher-level Language descriptions and Metacognition and Critical Thinking. If the categories themselves overlap, a perfectly consistent group could still be consistently measuring partly related things. The final ratings are therefore better understood as a collective, documented judgement than as direct observations of an objective capability quantity.
The important interface in this process is between the human benchmark and the model ensemble. The benchmark supplies an anchor for judging whether model outputs are close to the intended interpretation. It does not turn the models into independent experts, and the models do not replace the conceptual work of defining the indicators. They provide a way to carry a calibrated interpretation farther than human rating alone could practically reach.
Use models for scale, not for authority
After the 40-occupation pilot, the OECD assessed whether large language models could extend the mapping credibly. Four model families—ChatGPT, Claude, Gemini, and Mistral—rated the benchmark occupations under a harmonised protocol. Each model repeated the exercise five times, yielding 20 runs in total. The results were compared with the human mean for the same occupations, both to examine stability across runs and to see how closely each model tracked the benchmark.
The relevant design choice was aggregation. A single model can reflect model-specific preferences, prompt sensitivity, or random variation. Repeated runs show whether a rating shifts when the same task is presented again; multiple model families show whether a result depends heavily on one model’s style. In the OECD’s tests, the inter-model average aligned most closely with the human benchmark overall. The OECD therefore used that average to extend the ratings to the full space of 879 occupations.
This is a scalability argument, not an argument from artificial authority. The model ensemble makes it feasible to score a broad occupational classification with a common vocabulary and a reproducible procedure. It does not establish that a generated score is true in the same way that a directly observed measurement might be. The models were given occupational descriptions and tasks; they were not observing workplaces, interviewing workers, or watching how responsibility is distributed in practice. Their outputs inherit the framing and omissions of the supplied information.
The validation checks are accordingly layered. First, the OECD inspected rankings within each capability domain. Occupations near the top of Language tended to be communication- and knowledge-intensive; those near the top of Manipulation tended to be manual, technical, or operational. Relationships between domains also looked plausible: language, metacognition, and knowledge demands tended to travel together, as did Vision, Manipulation, and Robotic Intelligence. These checks ask whether the map has a coherent shape rather than producing arbitrary orderings.
Second, the ratings were compared with established occupational patterns, especially the distinctions already represented in O*NET. Broad alignment between cognitive-communicative occupations and manual-technical occupations provides useful external reassurance. It suggests that the model-based extension is recovering recognisable structure rather than inventing an entirely idiosyncratic classification.
Neither check answers every important question. A map can reproduce familiar occupational categories and still misjudge an outlier. It can be internally coherent while reflecting a mistaken assumption shared by the prompt, the source descriptions, and the models. This is why the report identifies direct review by occupational experts, workers, and employers as a remaining validation step. People who perform or manage an occupation can identify tasks that a generic description makes invisible, challenge implausible scores, and distinguish a capability that is formally listed from one that is central to real performance.
The need for that review is clearest when the occupation itself is moving. Occupations are not sealed containers. Technology, regulation, organisational design, and social expectations can change which tasks are bundled under a job title. During the human rating exercise, participants struggled with whether to score what a person does today or what an AI system would need to do to replace that person after a plausible future reorganisation. The OECD generally chose to anchor the rating in current task performance, leaving the question of displacement for subsequent analysis. That choice gives the current map a stable reference point, but it also means the map should not be mistaken for a forecast of the occupation’s future definition.
The same issue appears in the capability scales. Some levels have detailed examples; others are more abstract. A rater may follow the text of a level closely, or use relative comparison—giving a higher score to an occupation that seems more demanding than another. Both approaches can be reasonable in context, but they do not produce identical judgements. The scales may also have uneven spacing: a one-level movement in Language need not represent the same substantive change as a one-level movement in Metacognition or Manipulation. Once those values are added into a composite, the arithmetic looks more precise than the underlying meanings are.
The OECD’s revision history illustrates a different kind of uncertainty: updateability. The July 2026 revision corrected the assessed current level of AI vision capability to 3 and made small changes to affected vision figures and overall gap values. The report’s main interpretation remained stable, but some numbers changed. That is not an embarrassment to hide; it is evidence that a frontier measure can be corrected as its evidence base improves. It also means that a score should be read with its version and assumptions in mind, especially when comparing results published at different times.
This distinction between updateability and instability matters for workplace decisions. An updateable measure can be useful precisely because it is not frozen. If AI improves in visual inspection, a vision-related gap may narrow without implying that social or physical gaps have changed. If an occupational description is revised, the demand side may change independently. A manager should therefore treat the map as a current decision aid for where to investigate, not as a permanent rating of a role. A worker should not infer that a small numerical revision changes their personal prospects overnight.
The map also cannot reveal the complete chain from capability to outcome. It does not observe the cost of a system, the reliability required in a regulated setting, the availability of data, the time needed to integrate it, or whether workers have authority to alter a process. Those factors belong to adoption and organisational change. The mapping process can make a capability mismatch visible; it cannot decide whether an organisation will act on it.
This is the central trade-off in the OECD design. Human rating offers contextual judgement but does not scale easily. Model rating offers scale and repeatability but depends on the benchmark, the prompt, the occupational descriptions, and validation. Consensus can reduce avoidable disagreement but cannot resolve every conceptual overlap. External coherence can expose implausible patterns but cannot prove that every occupation has been represented correctly. The method is strongest when these components are treated as interfaces with different responsibilities rather than as a single source of truth.
Once those interfaces are understood, an exposure ranking becomes easier to read—and easier to question. A score reflects a chosen capability vocabulary, a mapped occupational profile, a current AI frontier, and a particular aggregation rule. Change any of those inputs and the meaning of “high exposure” can shift. The next question is therefore not simply which occupations sit at the top or bottom of the table, but what the table counts, what it leaves out, and why two reasonable exposure measures can tell different stories about the same work.