Where the Failures Land: Missed Findings, Distorted Values, Wrong Labels
This chapter examines the concrete failure surfaces of language-dependent medical AI: what a system loses when its output is still fluent enough to look acceptable. The focus is on omission, false-negative bias, subtype misclassification, and distorted structured values in radiology, ophthalmology, urgent questions, and ICU work. The reader should come away able to distinguish a merely lower score from a clinically directed failure and to judge why urgency and consequence must shape evaluation.
A report can be accurate on average and wrong where it matters
Radiology shows why a plausible document can conceal a serious problem. A model that generates a chest X-ray report is not being asked only to write grammatical sentences. It must identify findings in an image and preserve them in a report that another clinician may use. If the report omits a critical abnormality while sounding coherent, the error is a false negative: the output fails to mention something that is present.
The cross-lingual study found a pattern consistent with this risk. In the tested MIMIC-CXR conditions, overall accuracy could stay relatively near the English result while macro-F1 fell from 0.2938 to between 0.2149 and 0.2424. Macro-F1 is useful here because it does not allow performance on common classes to completely dominate the picture. It gives a clearer view of how the model handles individual findings, including findings that may be less frequent but clinically important.
The difference between these measures is not a technical footnote. Imagine a collection of reports in which most images contain findings that are easy to recognize or routinely absent. A system can be correct on many of those broad decisions and still become less reliable at naming a particular pathology. Overall accuracy may barely move because the missed finding is a small part of the total. A class-balanced measure can fall because the model’s performance has deteriorated precisely where the clinician needs sensitivity.
The study reports that the loss was concentrated on pathological content rather than merely on wording. The authors describe reports that could remain globally acceptable while selectively suppressing findings such as pneumothorax or consolidation. That is why “the report sounds right” is not a safety check. Fluency can hide a missing fact, and a missing fact can be more consequential than an awkward sentence.
The danger is easy to underestimate because omission often leaves no visible contradiction. An invented abnormality may attract attention; a missing abnormality can allow the rest of the report to read naturally. A clinician reviewing only the prose may see the expected structure, familiar terminology, and a sensible conclusion without realizing that a critical finding never entered the summary. That is why a safety endpoint should compare the output against the clinically relevant findings in the input, not only against the general tone or readability of the report.
This distinction also guards against a misleading explanation of the metric gap. A lower macro-F1 does not tell us by itself whether the model failed at visual recognition, language-specific interpretation, report prioritization, or some combination. The benchmark establishes a pattern of selective loss; it does not isolate one causal stage. Further testing would need to compare the image, intermediate extraction where available, and final report across the same language and inference conditions. Until then, the defensible conclusion is about the failure surface, not an unproven root cause.
The state diagram separates four things that are often collapsed into one. The clinical input is not the same as the model’s processing. A plausible output is not the same as a complete output. A detected error is not the same as an undetected error. Finally, review is not an abstract promise: it is the point at which someone must decide whether the information is complete enough to use, correct it, or reject it.
This pattern also helps distinguish symptom from cause. The symptom is a report that remains readable while missing a finding. The observed contributing condition is a language-dependent comparison in which the pathology-sensitive metric worsens more than the broad score. The clinical consequence is the possibility of false reassurance. The study does not, by itself, prove one single mechanism for why the omission occurs. It shows that the omission is the kind of failure a multilingual safety test must be designed to find.
A careful evaluator should therefore inspect more than the final report. They should compare the findings that were present in the image with the findings the report preserved, separate routine from critical pathology, and check whether the same pattern appears across language pathways. If the test reports only a global score, it may be measuring the system’s ability to produce acceptable-looking text rather than its ability to complete the clinical task.
The highest-risk details are the easiest to average away
The same diagnosis applies beyond radiology, but the vulnerable detail changes with the workflow. In ophthalmology, the critical distinction is not simply whether the model assigns a risk label. A broad high-risk category can remain intact while the system loses the disease subtype that changes the urgency of care. The study found that recall for angle-closure glaucoma declined in several non-English settings. Angle-closure disease is clinically time-sensitive, so a test that reports only broad high-risk accuracy can miss the relevant deterioration.
This is a problem of specificity, not just correctness. A broad label may be useful for an initial screen, but it cannot automatically substitute for a subtype decision. The evaluator must ask what action follows the label. If the next action is urgent assessment for a particular condition, then recall for that condition belongs in the safety endpoint. A model that preserves “high risk” but confuses the subtype may look stable to a general metric while becoming less dependable at the decision that matters.
The distinction is also important for review. A clinician may be able to correct a vague or incomplete draft if the underlying evidence is displayed, but a broad label can conceal the need to look more closely. If the workflow presents only the model’s category and not the details that support the subtype, the human reviewer has less opportunity to detect the error. The evaluation should therefore ask not only whether the label is correct, but whether the workflow gives the reviewer enough information to challenge a plausible but insufficient label.
The ophthalmology result also shows why the input configuration changes the failure surface. The study compared a two-type setting using textual case descriptions and tabular information with a three-type setting that added fundus photographs. Adding the images amplified the language gap. That does not mean images inherently make every model unsafe, or that the language is the sole cause. It means that the combination of language and visual evidence introduced a condition in which the tested system’s performance changed. A text-only result cannot certify the image-supported workflow.
ICU extraction exposes a different vulnerability. The system must preserve values from structured clinical information and use them in a risk judgment. Coarse fields such as age and sex were more stable in the reported analysis, while actionable parameters related to oxygenation, ventilator settings, vasopressor support, and acid-base status were more vulnerable. A system can therefore appear competent because it repeats the easy fields while quietly changing, smoothing, substituting, or omitting the continuous values that influence a decision about re-intubation or extubation safety.
This is not the same failure as a wrong glaucoma subtype or a missing radiology finding, even though all three may be described as “accuracy errors.” The ICU problem can begin before reasoning about the final decision: the model may be reasoning from an incorrect extracted value. If the output is fluent and the requested answer is binary, the underlying extraction error may be difficult to notice. The evaluation must compare the values in the source record with the values used in the model’s explanation, not merely check whether the final yes-or-no label looks plausible.
Structured data create a particular temptation to trust the system. Numbers, units, and named fields look precise on the screen, so a reviewer may assume that extraction has already been settled and focus only on the recommendation. But precision in formatting is not evidence of precision in content. A value can be copied into a fluent explanation with the wrong meaning, smoothed into a less alarming description, or omitted while the model still produces a decisive answer. The harm-relevant test has to follow the value from the source record to the model’s interpretation and then to the final decision.
Conversational medical questions add urgency as a further dimension. In HealthBench, emergency-referral scenarios showed larger language gaps than the overall task. This means that a single average score can mix routine questions with cases in which delayed or incorrect guidance carries greater concern. The result does not establish a direct patient outcome, and it does not mean every urgent question fails. It does show why the evaluator should separate urgent and routine subsets instead of allowing the routine majority to define the system’s apparent safety.
Risk-weighted evaluation is not a request to ignore ordinary cases. Routine performance still matters because real workflows contain both routine and urgent work. The point is to prevent an average from hiding a subgroup that carries disproportionate clinical consequence. An evaluator can report overall performance and still make the critical subset visible. The mistake is to treat the first number as a complete description of the second.
The same rule applies when the workflow has several stages. An urgent case may be lost during language interpretation, during extraction from an image or table, during the choice of a diagnostic label, or during the handoff to a reviewer. Measuring only the final answer tells the evaluator that something went wrong, but not whether the system preserved the information that a human needed to intervene. Subgroup analysis and field-level checks make the failure more actionable because they connect the observed error to the point at which the workflow stopped carrying the required information.
Across these examples, the failure is clinically directed in an important descriptive sense. The errors are not reported as random changes distributed equally across every piece of output. They cluster around findings, values, labels, and scenarios that clinicians need for decisions: pathology in a report, an urgent disease subtype, actionable physiology, or emergency referral. That distribution does not prove a single causal mechanism, and it should not be turned into a claim about the intrinsic difficulty of a language. It does show that the safety question must be framed around what the workflow is trying to preserve.
The study’s model comparison reinforces that boundary. GPT-series results showed a broad resource-ordered decline in the tested tasks, but Qwen3 did not reproduce the same pattern in every glaucoma setting. This matters for interpretation. The evidence supports testing each deployed model and workflow; it does not support ranking languages as inherently safe or unsafe, or assuming that one model’s failure profile transfers to another. A failure diagnosis that skips the model and task conditions is too broad to guide deployment.
A clinically meaningful evaluation therefore asks four linked questions. What information was present in the input? What did the system output or omit? Which errors change the next clinical decision? And how likely is the intended reviewer to detect and correct them? Those questions turn a score into a failure analysis. They also prevent a team from calling a system “almost as accurate” when the small difference is concentrated in the very findings that carry the greatest consequence.
The practical conclusion is restrained but consequential. Language-dependent medical AI risk is not adequately described by the amount of text that remains grammatical or by the percentage of answers that are broadly correct. It must be described by what the system preserves at the point of clinical use. Once omission, subtype loss, and structured-value distortion are made visible, the evidence from different workflows can be combined into a conditional risk profile rather than another undifferentiated score.