Guardrails That Check More Than Grammar
An AI-drafted patient-portal message needs an independent check that can notice clinical omissions, workflow problems, and mismatches that fluent language hides. The question is what such a guardrail should evaluate and whether local clinical precedent can make its judgments more useful. The available retrieval-augmented study offers a concrete answer: define errors in clinically meaningful terms, compare the draft with relevant context, and treat the automated result as a signal for accountable review rather than as a clinical verdict.
This distinction matters because ordinary measures of text quality are poorly matched to patient communication. Two messages can use similar words while differing in urgency or safety. A draft can be longer, more grammatical, or closer to a reference answer and still fail to tell a patient what to do. Conversely, a concise reply can differ substantially from a clinician’s usual phrasing without being clinically wrong. The guardrail’s job is not to reward stylistic resemblance. It is to make the failures that matter to patient action easier to identify.
A taxonomy turns “quality” into questions a checker can ask
The retrieval-guardrail preprint begins with a simple problem: a reviewer cannot reliably evaluate a message against a concept as vague as “quality.” The researchers and physicians first proposed a set of common discrepancies in AI-generated replies. They then used an inductive coding process on a stratified sample of 1,000 patient-message, draft, and clinician-reply triplets. A language model suggested additional patterns, a board-certified physician reviewed the original and new codes, and further manual refinement reorganized overlapping or overly broad categories.
The resulting taxonomy contains five domains, 24 subdomains, and 59 granular error codes. The exact inventory is less important here than the distinction it creates. It separates problems in clinical reasoning from problems in communication quality and readability, while also representing issues such as incomplete responses, workflow violations, inappropriate recommendations, and privacy or bias concerns. A message can therefore be evaluated on the question that matters: is the patient’s clinical problem represented and addressed adequately, or has the draft merely produced plausible prose?
This structure changes the unit of review. Instead of asking whether the draft resembles the final human reply, the checker can ask whether it omitted a material part of the patient’s question, gave an inappropriate next step, missed an escalation, or created conflicting instructions. The categories do not decide whether a message should be sent. They give a reviewer and an evaluation system a shared vocabulary for discussing why the message may be unsafe or unsuitable.
The preprint uses that taxonomy in a two-stage evaluation. First, an LLM judge receives the patient message, the generated draft, and the structured electronic-health-record context available at the time of drafting. It decides whether an error is present and provides a short explanation. If it identifies a possible error, a second stage assigns one or more structured categories, with confidence and justification. This separation is useful because detecting a problem and naming the problem are different tasks.