Overview
This series uses NIST’s TEVV-Athlon Framework to trace an AI claim from intended use and measurement target through controlled, realistic, and cryptographic evidence to a conditional deployment decision. It shows why a sound audit may remain local, how adaptive providers can exploit known samples, and why independent sampling, uncertainty, validation, and monitoring are needed; conclusions reflect NIST’s Initial Public Draft and a version 1 preprint.
Learning outcomes
- An audit pass establishes only the predicate tested on the specified model, sample, and conditions.
- NIST’s TEVV-Athlon keeps goals, measurement targets, Events, Tools, and decisions connected.
- Benchmarks, red teaming, user testing, and field testing expose different behaviors and should complement one another.
- Commit-sample-prove limits adaptation to known audit data, but sampling error, construct validity, and operational change remain.
- The cited cryptographic study supports a specific adaptive-data-forging diagnosis, not a claim that every audit or detector fails.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
What does passing an AI audit actually prove?
It proves that the specified model satisfied the stated predicate on the specified audit evidence and conditions; it does not by itself prove deployment suitability.
How does the NIST TEVV-Athlon connect an AI goal to evidence?
It moves from an intended-use goal to Metrology Blocks, then to Events and Tools, and finally synthesizes the evidence for a decision.
Can cryptographic certification validate an AI model for deployment?
Not by itself: it can bind a model and verify a stated computation or empirical predicate, while deployment validation also needs valid, relevant, sufficiently independent evidence and ongoing controls.
How can an AI audit reduce the risk of an adaptive provider?
Commit-sample-prove has the provider commit first and the auditor select fresh evidence afterward, then quantifies sample-to-population error; it does not ensure operational representativeness.