From Validation to Lifecycle Assurance

The central subject is the deployed clinical AI system and the question is whether it remains fit for its role after the conditions that supported validation begin to change. The evidence supports a precise answer: validation-era performance is not guaranteed to persist, but neither does every deployment fail in the same way or at the same rate. A defensible judgment must connect the operating context, the model's behavior, the workflow's response, the timing of available evidence, and the organization's willingness to act.

Validity is conditional, not permanent

A validation result is a statement about a relationship observed under particular conditions. The model received inputs with particular definitions and timing. Outcomes were labeled in a particular way. Clinicians and information systems produced and delivered the data through a particular workflow. The result may be strong evidence that the system performed well in that setting and period. It is not evidence that the same relationship will remain unchanged after the system becomes part of a changing clinical organization.

That distinction answers the title's claim without turning it into a slogan. Clinical AI does not “stay valid” automatically after deployment in the sense that validation does not preserve its own assumptions. The PLOS study found that four systems met predefined validation criteria and later diverged from their validation-era behavior even though their parameters, feature definitions, thresholds, and intended uses were not intentionally changed. The divergence was progressive, and calibration often changed before discrimination. The exact quantitative pattern came from one retrospective study of anonymized systems in one organization, so it is not a universal failure rate. But the repeated structure is important enough to change how continued use should be justified.

The mechanism is the relationship among context, input, output, and action. A workflow can change what information is available or when it arrives. The model can then apply an unchanged rule to a different information state. Its output can still look plausible and may still rank cases reasonably, while its absolute risk estimates become less reliable. A clinician or operational team may act on those estimates, changing both the downstream workflow and the outcomes later used to evaluate the system. The deployment has become a feedback system rather than a one-way calculation.

This chain also explains why no single metric can carry the entire assurance burden. Calibration asks whether predicted probabilities still align with observed frequencies. Discrimination asks whether higher-risk cases remain ranked above lower-risk cases. Operational telemetry asks whether inputs, inference, and output delivery are functioning under expected conditions. Impact evidence asks whether the output still produces the benefit that justified the deployment and whether unintended burdens or harms are appearing. These are related questions, but they are not substitutes.

A system can therefore be “working” in one sense and failing in another. It can be available while its calibration has drifted. It can be accurate on a benchmark while the output arrives too late to influence the decision. It can be well calibrated while the intended team ignores the alert or lacks capacity to complete the downstream action. The appropriate judgment is not a single label such as safe or unsafe detached from context. It is whether the evidence available for this deployment is adequate to support continued use for this role, at this time, with these controls.

The minimum chain for ongoing assurance

A decision maker reviewing a deployed clinical AI system should be able to follow a continuous chain rather than inspect isolated reports.

It begins with the operating envelope. What population, workflow, data timing, outcome definition, threshold, and intended action did validation represent? The point is not to reproduce every historical detail. It is to identify the assumptions whose failure would change the meaning of the output.