A Medical AI Safety Test Is a Workflow Map, Not a Single Score
This chapter turns language-dependent medical AI safety into an evaluation map. The focus is not simply how many languages a model can answer in, but how language interacts with the task, input type, reasoning route, output, clinical risk, and human reviewer. The practical result is a minimum vocabulary for reading a safety claim: the reader should be able to ask what was tested, what could go wrong at each interface, and which endpoint would actually expose that failure.
A workflow has more than a language setting
A claim that a model “works in Persian,” “supports Thai,” or “is multilingual” is incomplete until the workflow is named. The language is the setting in which information enters and leaves the system, but it is only one part of the path. A useful evaluation map connects at least seven elements: the deployed language, the model family, the inference path, the clinical task, the modality of the input, the form of the output, and the person or process responsible for review.
The cross-lingual clinical study makes this distinction visible by using four different kinds of medical work. HealthBench represents open-ended conversational medical questions. The system must understand a request, identify relevant clinical considerations, and produce advice that follows a rubric. That is not the same operation as MIMIC-CXR, where the model combines chest images with report generation. In the first case, the important information is carried mainly through language and clinical context; in the second, language is joined to visual interpretation and the obligation to mention findings in a report.
The ophthalmology task adds another variation. It was evaluated with textual case descriptions and tabular data, and also with fundus photographs. The latter configuration asks the model to coordinate visual and linguistic information rather than simply read a written case. The ICU task adds time-sensitive physiological values and asks the system to extract and interpret information relevant to re-intubation risk. A model can therefore face different failure opportunities even when the prompt language is unchanged. It may understand the words but misread a number, preserve a general diagnosis but lose a time-critical subtype, or produce a polished report that omits an image finding.
The model and the inference path must be recorded alongside the task. The study used GPT-5.2 for HealthBench after GPT-4o produced near-zero non-English scores under the strict rubric, GPT-4o for the other main benchmark tasks, and Qwen3 VL 235B A22B Thinking as a multilingual reference model. This is not a fully crossed experiment in which every model performs every task under identical conditions. That limitation is important: a difference between two results may reflect model family, task, language, inference path, or their interaction. It cannot automatically be assigned to language alone.
The inference path is especially easy to hide. Native-language reasoning keeps reasoning and response in the target language. An English-constrained route asks the model to reason in English while returning the answer in the target language where permitted. A back-translation route first translates the target-language input into English before the model reasons and answers. These are different systems from an evaluation standpoint. They may change terminology, prompt alignment, intermediate reasoning, and the opportunities for translation error. Calling all three outputs “the Thai result” erases a decision that may affect safety.
A useful map also records what the system is expected to do with the output. Is it answering a patient-facing question, drafting a report for a specialist, extracting facts into a structured record, or assigning a category that triggers a next step? The wording of the output is not the only concern. A report can be judged for whether it preserves every critical finding; a structured extractor can be judged for whether it preserves numbers, units, and clinically important polarity; a classifier can be judged for whether it separates urgent subtypes. The same language error can have different consequences depending on the output’s destination.
This is why the study’s four tasks should be read as contrasting workflow shapes, not as four interchangeable tests. Conversational care places more weight on context, safety advice, and appropriate escalation. Image-to-report generation places weight on translating visual evidence into a complete written account. Ophthalmic diagnosis tests the interaction between records and images. ICU extraction tests whether a system can carry precise physiological information into a risk judgment. Each task exposes a different interface between language and medicine. A test that omits one of those interfaces may still be valuable, but its conclusion must stay within that boundary.
A team does not need to test every imaginable combination before learning anything. It does need to name the combinations it has chosen and the risk that made them worth choosing. For a system intended for image-supported diagnosis, adding the image condition is not optional merely because a text-only prototype performed well. For an ICU documentation tool, checking actionable continuous values matters more than adding another low-stakes demographic field. The map is a way to allocate scarce evaluation effort according to the harm pathway rather than according to what is easiest to score.
The diagram is not a software architecture. It is a boundary map for evaluation. The language enters a model through a particular path, but the clinical workflow determines what the output must accomplish. The modality determines what information must be combined. The safety endpoint translates a general aspiration such as “accuracy” into a question about missed pathology, preserved values, or usable documentation. The reviewer owns the final judgment about whether the output is safe enough for the intended role.
This map also clarifies what changes hands at each interface. A developer controls the selected model and inference route. A deployment team controls how a language is presented, how inputs are translated or formatted, and where the output appears. A clinician may control whether a draft is accepted, edited, or discarded. A safety evaluator controls neither the patient’s risk nor the clinical reality, but can choose whether the test measures a meaningful consequence. When those responsibilities are blurred, a high score can be mistaken for permission to reduce review.
The ownership question is practical. If a translated input loses a negation, the problem may belong to the translation stage rather than to medical knowledge. If the input is preserved but the model omits a finding in the report, the model-and-task interface needs investigation. If the report contains the finding but the display hides it among boilerplate, the clinical application has introduced another risk. If a clinician is expected to catch the omission, the evaluation must establish that the omission is detectable under realistic time and information constraints. A workflow map makes these possibilities distinguishable instead of assigning every failure to “the language model.”
The map also prevents a different mistake: assuming that adding a reviewer automatically makes an unsafe system safe. Review is a control with its own limits. It can detect some errors, but it consumes time and depends on the reviewer seeing the relevant input, understanding the task, and having authority to correct the output. The prospective documentation evidence shows why correction burden belongs in the map. A draft that needs extensive substantive editing may still have a role, but it should not be presented as equivalent to an output that can be accepted with minimal review.
Choose the endpoint that matches the harm
Once the workflow is mapped, the next question is not “which score is highest?” It is “what kind of mistake would matter here, and does the endpoint make that mistake visible?” A general accuracy value may be useful, but it can be too coarse for a safety claim. Different endpoints expose different parts of the system’s behavior.
Consider radiology report generation. If most examples are dominated by common or unremarkable findings, a model may obtain a tolerable overall accuracy while failing on the less frequent findings that matter most. Macro-F1, which gives individual classes more balanced influence, can reveal this pattern. In the cross-lingual study, overall accuracy stayed relatively near the English condition in some radiology comparisons while macro-F1 fell from 0.2938 to between 0.2149 and 0.2424. The point is not that macro-F1 is always the correct clinical endpoint. The point is that a safety evaluation must choose a measure that can see the failure it is worried about. The choice is especially important when a finding is uncommon. A model can appear strong by correctly repeating many negatives while missing the smaller set of positive findings that clinicians need to act on. In that setting, the evaluator should inspect false negatives and the recall of the critical class, not let the prevalence of routine cases make the result look safer than it is.
This does not mean that one metric should replace all others. Precision matters when false alarms create harmful workload; recall matters when omission is the greater danger; calibration matters when a score is used to prioritize review; and field-level correction matters when the system drafts a record. The endpoint is a statement about the intended use. If the deployment team cannot explain why a metric represents the clinical risk, the metric is probably being used because it is convenient rather than because it is relevant.
For ICU extraction, the relevant question is not whether the system preserves easy fields such as age or sex. The study found that actionable parameters—including values related to oxygenation, ventilator settings, vasopressor support, and acid-base status—were more vulnerable in the tested non-English conditions. Actionable-parameter recall is therefore more informative than a broad field-average score. A system that gets many low-stakes fields right but misses a value that influences extubation safety should not be described as equivalent to the language baseline.
Glaucoma classification presents a related distinction between a broad label and a clinically important subtype. The study reports that angle-closure recall declined in several non-English settings. A model might still preserve a general high-risk category while misclassifying the specific condition that requires urgent attention. The endpoint must therefore reflect the decision a clinician actually needs to make. If the workflow depends on identifying a time-critical subtype, subgroup recall belongs in the primary safety view, not in a footnote after average accuracy.
The same principle applies to conversational care. Emergency-referral questions showed larger language gaps than the overall HealthBench task. A general score that combines routine and urgent cases can dilute the very subgroup in which a failure has the greatest consequence. Risk-based evaluation does not mean discarding average performance; it means refusing to let the average stand in for every clinical situation.
Images, tables, and time series require their own checks because they create additional interfaces. In the ophthalmology task, adding bilateral fundus images amplified the language gap relative to the text-and-table configuration. The model had to combine visual evidence with language-specific interpretation. In radiology, it had to convert image understanding into a report in which omissions matter. In ICU work, it had to preserve structured values while explaining a clinical decision. A text-only test can tell us how the model handles translated prose; it cannot certify the full workflow that uses images or physiological data.
Correction burden is another endpoint for a different kind of risk. In the prospective documentation study, physician review measured whether a generated record needed edits and how many field slots required correction. This does not replace diagnostic accuracy or patient outcomes. It measures operational usability: how much clinical work remains before the output can be used. That distinction matters because a system may be acceptable as a draft assistant with close review but unacceptable as an automated record generator. The endpoint should match the proposed level of reliance.
A minimum safety claim should therefore name the model, language, inference path, task, modality, endpoint, important subgroup, and reviewer. “The model supports five languages” says almost none of this. “The model generated radiology reports from chest images in four non-English languages under native reasoning, with critical-pathology F1 assessed against an English reference and physician review of the output” is much more informative, even before the result is reported. It tells the reader what kind of evidence is available and what remains outside it.
The evaluation map does not make testing infinite or impractical. It makes omissions visible. A team may have limited resources, but it can still state which combinations were prioritized and why. Critical findings, urgent cases, actionable values, and workflows with little human review deserve more attention than low-consequence text polishing. The central discipline is to connect the endpoint to the harm pathway instead of letting the easiest metric define the safety claim.
That connection will matter in the next stage. Once a workflow is decomposed into interfaces and endpoints, the important failures become easier to see: a missing radiology finding, a distorted physiological value, or a wrong disease label can be followed back to the part of the configuration where risk surfaced. The next chapter examines those failures directly, including why an output can remain plausible while becoming less clinically complete.