A Safer Standard for Medical AI Agents: Evaluate the Process, Not Just the Answer
The central question for medical Agentic AI is not whether a system can generate an impressive answer. It is whether the system can pursue a defined clinical goal through an appropriate, observable, and bounded workflow—and whether the available evidence justifies the role people want to give it. This series has supplied the pieces for that judgment: a model of agentic execution, a map of architecture and failure interfaces, a layered evaluation stack, and a case showing how expert expectations can be reused at scale. Together they support a disciplined conclusion: a final output is one observation from a clinical workflow, not proof that the workflow was safe or useful.
The unit of trust is the workflow
A medical agent should be judged at the level of the claim being made about it. “The system can produce a report similar to a reference” is one claim. “The system can select the right evidence, recover from tool failures, reduce clinician work, and support a decision in a live setting” is a much larger claim. The second cannot be established by improving the first metric.
The difference is causal. An agentic system may interpret a goal, break it into subtasks, retrieve information, call a specialized model, maintain intermediate state, revise its plan, and present an output for a human decision. A failure in any transition can alter the final result. Sometimes the final answer will be wrong. Sometimes it will be correct by chance. Sometimes the answer will be clinically reasonable but reached through a path that cannot be trusted on the next case.
This makes the execution trace part of the object under evaluation. The trace should answer a small set of practical questions. What was the intended task? What information was available? Which action or tool did the agent choose, and why was that action permitted? What input crossed the interface? What result came back? How did the result change the working state? Did the system detect uncertainty, conflict, or failure? Did it revise, abstain, escalate, or proceed? What did the clinician accept, modify, or reject?
These questions do not require treating a hidden chain-of-thought narrative as an audit record. They concern observable events and relationships. A record of the selected tool, its input and output, the retrieved source, the intermediate state, and the human intervention is more useful for accountability than a fluent explanation that cannot be independently checked.
The six evaluation dimensions from the medical-agent review can be understood as different tests of the same workflow. Task performance asks whether the output met the intended target. Process reliability asks whether the system took an appropriate path and handled its tools and intermediate results correctly. Evidence traceability asks whether the sources, observations, or computed values actually support the claims. Safety asks whether uncertainty, permissions, failure handling, abstention, and escalation were appropriate. Clinical utility asks what the system did to time, revision burden, overrides, trust, and final decisions. External validation asks whether those observations persist across settings rather than belonging only to the original data and institution.
No dimension can silently stand in for another. A strong report-similarity score does not show that a clinically important omission is rare. A successful tool call does not show that the tool was needed. A citation does not show that the source applies to the patient. A calibrated confidence estimate does not show that the system chose the right action. Faster drafting does not show that the total verification burden fell. Performance on a retrospective dataset does not show that the agent will behave safely when a live workflow is interrupted or information is missing.
The evaluation target should therefore be a structured claim, not a vague adjective such as “robust” or “trustworthy.” The claim should identify the task, the user, the data and setting, the degree of autonomy, the action boundary, and the consequence of error. It should state which dimensions were measured and which were not.