The Workflow Around the Model Is Part of the Model's Risk
The deployed clinical AI system in this chapter is treated as a participant in clinical work, not as an isolated calculator. The focus is the chain by which care activity becomes documentation, documentation becomes model input, and model output becomes a decision. Tracing that chain lets us see how an unchanged model can receive information at a different time, in a different form, or with a different meaning—and why those changes are warning signs without automatically proving harm.
Clinical work becomes model input through a chain of actions
A clinical data field can look like a neutral fact after it has been stored. Before storage, it is usually the result of a sequence. Someone decides to order a test, a patient provides a sample, a laboratory processes it, a result is released, a clinician or staff member records an observation, and an information system makes the result available to downstream applications. A note field may depend on a clinician remembering a prompt, selecting a template, or completing documentation before a shift ends. A medication record may reflect both a treatment decision and the timing of an administration event.
A model sees the result of these processes, not the processes themselves. It may be designed to use the most recent vital sign, a laboratory value, an order, a medication event, or a structured documentation field. If a clinical team changes when it records that information, the model's input distribution changes. If an interface changes which field carries the information, the feature may be absent even though the underlying fact exists elsewhere. If a value arrives after inference, the model is not seeing the same patient state that a retrospective analyst might see after the encounter is complete.
This is why the phrase “data quality” can be too vague. A missing field may reflect a broken pipeline, but it may also reflect a clinical choice, an uncompleted workflow step, or a value that was not yet available at the moment a decision had to be made. The same missingness rate can therefore have different meanings in different settings. An absent laboratory result in an acute-care model may mean that the test is pending; in another workflow it may mean that the test was not clinically indicated. A missing documentation element may signal an interface problem or simply a different way of recording the same work. The model cannot know which interpretation is correct unless the system around it preserves that context.
The framework supplied by the sources is sociotechnical in this precise sense. Clinical information is generated through interactions among clinicians, patients, order sets, templates, interfaces, policies, and software services. EHR upgrades can rename a note type or relocate a feature table. A revised protocol can change which tests are ordered. A new work queue can change when staff acknowledge an alert. A change in staffing or workload can alter how completely and how quickly documentation is finished. These are not merely administrative details when an AI system depends on the resulting data.
The path from work to input can be made explicit:
The diagram is not a claim that every clinical AI system has the same sequence. It is a way to locate the interfaces where a validated relationship can weaken. The output can change because the input changed, because the input arrived at a different point in the clinical episode, or because the system no longer delivers the output into the action pathway for which it was evaluated.
Three kinds of drift can overlap
The longitudinal study uses three analytical categories to describe change: temporal drift, population drift, and workflow-induced drift. They help organize investigation, but they should not be mistaken for three sealed compartments with one cause assigned to each observation.
Temporal drift describes performance changing over time in a way not attributed to an explicitly identified population or workflow shift. It is a description of the time pattern, not an explanation for it. A gradual decline across successive months may be real even when the organization cannot yet identify which process changed.
Population drift describes a change in patient demographics, case mix, or outcome prevalence. If the proportion of high-risk cases changes, a predictive system may face a different forecasting problem even when the data pipeline is technically stable. A change in patient mix can alter calibration, discrimination, or the usefulness of a threshold. The absence of a large demographic shift therefore does not rule out degradation, just as the presence of one does not identify the complete mechanism.
Workflow-induced drift describes changes in data availability, timing, or semantic meaning created by clinical practice, documentation, or system configuration. This category is especially important because it can operate within a single institution and remain invisible in broad population summaries. The patient population may look similar while the signals used to represent that population have changed.
These forms can coexist. A new care pathway may change which patients receive a test, changing both the population represented by the feature and the feature's availability. An EHR upgrade may change documentation timing at the same time that a seasonal shift changes case mix. A model's declining performance can therefore have several contributing conditions, and the categories are best used to decide what to inspect rather than to declare a single root cause.
The distinction also protects against an overly narrow idea of dataset shift. The problem is not only that the model sees different patients. It may see the same broad patient population through a different measurement process. A laboratory result that used to be present before inference may now arrive afterward. A structured field may be completed only after a clinician has already acted. A radiology-derived feature may depend on an imaging-processing step whose timing or availability changed. The statistical value of the field is inseparable from the workflow that makes it available.
Consider a deterioration-risk score that runs while a patient is being assessed. During validation, recent observations may usually have been recorded before the score was calculated. After deployment, a change in rounding practice or order timing may leave those observations pending at inference. The model still runs and may return a number in the expected format. Yet the number now summarizes a thinner information state. If the same change affects many encounters, the model can show a systematic shift without a single malformed record or a visible software outage.
The clinical meaning of that shift depends on what happened upstream. If missingness reflects a deliberate decision not to order a test, it may carry information about perceived risk or clinical priorities. If it reflects a result that is pending, it may signal the stage of an encounter. If it reflects a broken interface, it says more about system integrity than about the patient. Those interpretations can point to different responses. A clinical team may need to change a workflow, an integration team may need to repair a data route, or a governance group may need to question whether the model's intended use still matches current practice.
This is why investigating drift requires more than comparing a current patient histogram with an old one. The organization needs to ask when the input became available, which workflow step created it, whether the field was expected for that encounter, and what happened after the prediction. The PLOS analysis aligned clinical, outcome, and operational data at the inference level for precisely this kind of temporal interpretation. Even then, the study treated workflow-induced, population-related, and temporal categories as overlapping analytical lenses rather than as a definitive causal decomposition.
The distinction also explains the practical value of telemetry. Population indicators can tell an organization that the people arriving for care look different. Workflow indicators can tell it that the way information is created or delivered has changed, often in a place the organization can inspect and correct. That does not make telemetry more truthful in every setting. It makes it closer to an operational control point. A signal that points toward a modifiable interface or documentation process can support earlier investigation than a broad change in case mix whose clinical meaning remains unclear.
Why missingness and latency can carry meaning
Two operational signals are particularly instructive: missingness and latency. Missingness is the proportion of expected inputs that are unavailable when the model runs. Latency is the delay between the underlying clinical event or data creation and the moment the model can use the information. Both are observable near inference time, and both can change before a complete outcome label exists.
If a model expects a recent vital sign and that value is absent, there are several possible paths. The patient may not have been assessed; the assessment may have occurred but not been documented; the documentation may be stored under a different field; the data interface may have failed; or the value may simply not have been available yet. These possibilities are clinically different. A missing value can be a marker of workflow state, not just a blank cell to be imputed away.
Latency creates a related problem. A value arriving after inference may eventually appear in the record, giving a retrospective analyst the impression that the model had access to it. At the decision point, however, the model operated with less information. If the clinical team changes the timing of test ordering or documentation, a feature can retain the same name and nominal definition while representing a different information state at prediction time.
Feature availability is therefore a bridge between workflow and performance. A model may be mathematically unchanged, yet the information supplied to it can become less complete, less timely, or semantically different. The resulting score can shift even if the raw feature definitions in the model specification have not changed. In a threshold-based workflow, that shift can alter how many patients cross an escalation boundary or how much attention is directed to a queue.
The empirical evidence supports treating these signals as monitoring priorities. In system-specific models, higher input missingness was associated with lower calibration slope, with standardized coefficients ranging from -0.42 to -0.21 across the four systems. Data latency showed negative associations ranging from -0.35 to -0.08. By comparison, coefficients for demographic shift were small and inconsistent, ranging from -0.06 to -0.03. The numbers describe associations in this study, not universal effect sizes or a causal formula for correcting a model. Their value is comparative: operational changes were more consistently aligned with calibration degradation than the broad population indicator used in the analysis.
That result does not mean population change is unimportant. It means that monitoring only demographics or case mix can miss a failure surface that is closer to the way the system actually receives information. It also does not mean that every rise in missingness will damage performance. Missingness may be harmless for one feature, expected for one workflow, or a transient technical event. Its meaning must be interpreted in relation to the task, the timing of inference, and the downstream decision.
What an unchanged model can and cannot tell us
A frozen model does not provide a frozen interpretation. When the environment changes, the model still applies the same learned rule, but the rule may be applied to a different distribution of information. This is the mechanism behind the earlier observation that post-deployment divergence need not follow retraining. The model's stability can make the problem harder to notice because users and maintainers may assume that no change has occurred.
The telemetry findings therefore support an early-warning role, not a complete diagnosis. A rise in missingness or latency can justify checking an interface, a documentation step, an order pathway, a protocol, or a clinical process. It cannot by itself establish that the model's predictions are clinically wrong. The same signal can be a trigger for investigation, a mediator through which performance changes, or a correlate of another unmeasured change. Separating those roles is essential when deciding whether to repair the pipeline, adjust the workflow, recalibrate the model, or wait for outcome-based confirmation.
This is also why the two selected sources should be read as complementary rather than interchangeable. The empirical study reports stronger and more consistent associations for workflow telemetry in its four systems. The practical monitoring framework treats integrity and operational signals as one line of sight within a broader program that also includes performance and impact. Neither source says that telemetry replaces outcomes. Together, they support using near-inference signals to shorten the time to investigation while preserving slower, label-dependent reassessment.
The next question is how to interpret the resulting performance change. A system can continue to rank patients reasonably while its absolute risk estimates drift, and a label can arrive only after the model has influenced care. Understanding calibration, discrimination, and label timing is what separates a useful early warning from the stronger claim that the system has become unsafe.