Measuring Effectiveness Without Fooling Ourselves

This chapter asks what evidence can legitimately show that an age-assurance process is effective. The focus is the gap between activity and protection: completed checks, passing results, observed visits, reported exposure, and traffic changes each describe a different part of the system. By separating their units of analysis and causal strength, the reader can judge an age-assurance claim without mistaking a busy process or an impressive metric for a safer online environment.

The distinction matters because age assurance creates unusually visible numbers. A service can count every check it performs, every user it admits, every appeal it receives, and every request it blocks. Those counts are useful, but none is a complete measure of whether children are protected. The number of checks may rise because more people are being asked, because the same people are being checked repeatedly, or because a new service has been added. A passing result may reflect who was sent to the check rather than how well the method classified the whole user base. Measurement has to follow the process all the way to the outcome it is meant to produce.

A check count is an activity measure

Ofcom’s early UK assessment recorded more than 69 million completed age checks across 32 analysed services between July and December 2025, compared with roughly 3 million in the preceding six-month period. That large increase shows that age assurance was deployed at scale after the children’s-safety duties took effect. It does not, by itself, show that children encountered less harmful content. The total covers only the analysed services, excludes the estimates produced continuously by passive age-inference models, and includes a period in which some services had incomplete monthly data.

A useful first question for any number is therefore: what event is being counted? A completed check counts an interaction with a process. It may end in an adult result, a child-protection result, a retry, an abandonment, or an appeal. If the service retains the result against an account or device, later visits may not create new checks even though the protection remains active. A falling check count can then indicate less demand, successful reuse, or a change in deployment—not necessarily a decline in effectiveness.

Ofcom reports that all analysed services offered at least one method from its list of methods capable of being highly effective, with facial age estimation and photo-ID matching the most common, and that services deployed an average of three methods. Those facts describe availability and design choice. They do not establish that each method was used equally, that every configuration met the criteria, or that offering more methods improved the outcome for every user. A catalogue of options is not a performance evaluation.

The same caution applies to the share of users who pass. Ofcom found an average of 57.8% of users completing an age check were determined to be 18 or over across the analysed services, but the proportion varied significantly by service type. The figure cannot be read as a simple accuracy rate.