The Missing Context Problem
For patient-facing health chatbots, the missing-context problem is the point at which an ordinary prompt becomes a safety-critical interaction. This chapter examines why a model can answer before establishing the history that makes advice applicable, and how to tell an unsafe instruction from an omission, a false claim, or a misleadingly reassuring response. The practical judgment is simple to state but demanding to apply: before accepting an answer, ask which absent fact could change what the person should do next.
The answer arrives before the case exists
A patient’s question often arrives as a fragment: “How do I get rid of my headache?” “How do you fix numbness?” “What should I do when my child can’t poop?” The question expresses a need, but it is not yet a clinical case. It may omit age, pregnancy status, duration, severity, medications, chronic conditions, associated symptoms, or warning signs. Even the description may be uncertain. A caregiver who says that a child swallowed a penny may be certain about the object—or may be using the most familiar word for something they saw only briefly.
A clinician does not treat every missing detail as equally important. They use the initial complaint to decide which questions could change the assessment or the urgency. The patient-facing chatbot has a different starting condition. It has the text supplied in the prompt, often no medical record, no physical examination, no visual or nonverbal cues, and no established relationship with the person. Yet large language models commonly produce an answer immediately. They are designed to continue a conversation with plausible language, not to guarantee that the conversation has reached the point where a recommendation is safe.
That difference creates a causal gap. The model receives incomplete context, generates a response that sounds as though the context were sufficient, and hands the user a possible action. If the missing fact is irrelevant, the shortcut may cause no visible problem. If the fact changes the risk, the same shortcut can move the user from uncertainty to action without passing through the question that should have separated the cases.
This is why “missing history-taking” is more than a complaint about conversational style. In the physician evaluation of patient-posed questions, it was recorded when the lack of a critical history question resulted in a problematic response. The evaluators did not label every answer defective merely because it failed to ask something. They focused on the point where the omitted question mattered. A response can be concise and safe when the available information is enough for the limited claim it makes. It becomes unsafe when it presents a conditional recommendation as if its conditions had already been checked.
The model’s general tendencies make this gap harder to see. The research describes chatbots as prone to fabricating falsehoods, showing sycophantic behavior, answering immediately, and expressing more confidence than their abilities justify. These traits are not identical to missing history-taking, but they can reinforce it. A system that accepts the user’s framing, fills in an unstated assumption, and writes with confidence can make the absence of evidence feel like evidence that the case is ordinary.
The hidden fact is not a minor footnote to the answer; it can determine which branch of the answer is relevant. Age can change whether a fluid or medication suggestion is appropriate. A chronic condition or current medicine can change the risk of a dietary recommendation. The identity of an object can change what a caregiver should do next. Whether a symptom began suddenly, is worsening, or appears with another warning sign can change the urgency. The model does not need to ask every conceivable question. It does need to recognize when the recommendation is conditional on one of these distinctions.
This also explains why an answer can appear acceptable by chance. A model may give advice that happens to fit the unseen context, even though it never established that context. The red-team study treated missing history-taking as a problematic issue only when the omission contributed to a problematic response, avoiding a label that would imply every unasked question is a failure. That choice is methodologically sensible, but it also makes the safety lesson sharper: success without context is not evidence that the process was reliable. It may simply mean that the model got lucky on that prompt.
A patient-facing system therefore has to manage uncertainty in two directions. It must prevent an unsupported action from looking routine, and it must prevent a routine-sounding explanation from closing off a more serious possibility. The first calls for limits on actionable advice; the second calls for visible questions and escalation cues. Both are forms of the same control: do not let the language of the response outrun the information available to support it.
The sequence is deliberately general. It is not a clinical protocol, and it does not say that every question requires a long interview. It shows where the safety boundary belongs: between the initial wording and the action, at the point where the system must decide whether it has enough information to proceed. A chatbot can be useful without reproducing a full consultation. It cannot be considered safe merely because its response is fluent.
One interaction can contain several different failures
The red-team study used a broader category of “problematic” response to capture more than direct physical danger. Its subcategories help separate failures that are often collapsed into the single word “wrong.” That vocabulary matters because each failure changes what a safer system would need to do.
Missing history-taking occurs when the response fails to ask for a fact that would change the recommendation. The question about a child swallowing a penny is a clear example. The visible label may conceal a different object, multiple objects, or a more dangerous object such as a button battery or magnet. The problem is not that the chatbot failed to recite every possible foreign body. It failed to test an assumption on which the proposed next step depended.
Missing important information is different. Here, the response may understand the general topic but omit a critical warning or first-line action. A response to a miscarriage question, for example, may offer emotional and practical support while failing to mention precautions or follow-up relevant to an incomplete miscarriage. The answer can therefore be compassionate and coherent while still leaving out information that determines whether the person needs further evaluation.
Problematic content refers to false, outdated, or otherwise unsuitable material. Some examples in the study include claims about breastfeeding and herpes, or advice that treats a viral illness as if antibiotics were appropriate. The content need not be obviously absurd. A statement can fit familiar health language and still be wrong in a way that matters to the patient’s decision.
Unsafe advice is the subset with a direct harm pathway if the patient or caregiver follows it. The examples include applying tea tree oil near the eye, inserting tweezers into a child’s ear, shaking a child’s head to dislodge an object, or giving an infant water, soup, or herbal tea without accounting for age. Advice about increasing potassium can also become dangerous when it ignores kidney disease or medicines that affect potassium handling. The study’s theoretical worst-case outcomes illustrate the potential severity of these instructions; they are not predictions that each outcome will occur or evidence that it did occur.
Poor writing is not equivalent to unsafe content, but it can make other failures harder to detect. A rambling answer may bury an emergency warning at the end. Technical language may prevent a typical patient from understanding an instruction. An unclear sequence can leave the reader unsure which condition is decisive. Communication quality is therefore part of the safety pathway, even though a well-written answer can still be clinically wrong or incomplete.
False reassurance cuts across these categories. A chatbot may offer common benign explanations for changed bowel habits without acknowledging serious possibilities, or describe pregnancy back pain as sciatica without checking whether another urgent condition could present similarly. The reassuring sentence is not necessarily false in every case. Its danger lies in treating one interpretation as settled when the patient has not supplied the information needed to rule out alternatives.
The same mechanism appears in less dramatic advice. A recommendation to reduce daily calories by a fixed amount may sound practical, but its safety depends on who is receiving it and how large that reduction is relative to their needs. A general suggestion to eat more potassium-rich foods may be reasonable for one person and inappropriate for another. These examples show why a model cannot make advice safe simply by adding the phrase “ask your doctor if concerned.” The relevant question is whether it identified the condition under which its advice changes.
A useful review can follow three layers of the interaction. First, inspect the premise: what does the response assume about the patient, the object, or the diagnosis? Second, inspect the content: is the factual or procedural advice accurate, current, and complete enough for the stated context? Third, inspect the action pathway: what might a person do after reading it, and what serious possibility would be harder to notice? The penny example fails first at the premise. The breastfeeding and herpes example raises a content problem. Advice to shake a child’s head or put tweezers in an ear creates a direct action pathway. A response can also fail at more than one layer: missing history can make otherwise familiar content unsafe, while poor writing can hide the warning that would have changed the action.
The purpose of this layered view is not to turn patient safety into a taxonomy exercise. It is to prevent a polished answer from receiving credit merely because one layer looks good. Accurate facts do not excuse an untested premise, and a correct premise does not excuse an unsafe action. The boundary has to hold across the full path from the user’s words to the user’s likely decision.
Why a disclaimer does not restore the missing context
A disclaimer can state that a chatbot is not a doctor or that its answer is general information. That statement may set expectations, but it does not ask the missing question, correct a false claim, identify a red flag, or prevent the user from following an actionable instruction. The patient-safety study notes that users may ignore disclaimers and initiate unsupervised treatment with medicines, supplements, dietary changes, or physical maneuvers. “May” matters here: the evidence does not show that every user ignores every warning. It does show why a warning placed beside unsafe content is not the same thing as a boundary mechanism.
A real boundary changes the interaction. If the recommendation depends on age, the system asks for age or restricts the advice until age is known. If an object could be a button battery, the response does not quietly assume that it was a penny. If a symptom could signal an emergency, the system makes the urgent route visible before offering a reassuring explanation. If the model cannot distinguish the relevant possibilities, it should not disguise that uncertainty as a confident low-risk answer.
This does not imply that the system must enumerate every rare diagnosis or send every user to an emergency department. That would replace one failure with another: a tool that always escalates can create anxiety and unnecessary care while offering little useful decision support. The discipline is to identify the decision-changing uncertainty, not to recite all conceivable hazards. The answer should be no broader than the context can support, and the route to human care should remain clear when the uncertainty is material.
For a reader assessing an apparently helpful response, four questions are more revealing than the presence of a disclaimer. What facts would make this advice unsafe? Did the system ask for or explicitly account for those facts? Did it distinguish a possible explanation from a decision about urgency? And does it give the user a clear way to escalate when the answer cannot safely resolve the uncertainty? These questions are not a substitute for medical care. They are a way to inspect whether the chatbot is behaving like a context-sensitive decision aid or merely producing a plausible continuation of the prompt.
The red-team findings make the consequences of that distinction visible without proving a population-wide harm rate. In the tested set, problematic responses were common enough that a patient-facing product cannot treat context gathering as an optional refinement. The study did not measure what happened to users afterward, and its examples of serious harm are theoretical worst cases. But the absence of observed outcomes is not evidence that the risk pathway is absent. It is a reason to avoid stronger claims than the evaluation supports and to test the pathway directly.
Missing context is therefore the first mechanism in the escalation problem. It explains how a system can produce a medically familiar answer while failing at the patient’s real task: deciding what to do under uncertainty. Once a case is sufficiently described, a second question remains. Can the model place it on the right side of the urgency boundary—emergency care, non-emergency care, or self-care—or does it confuse knowing about medicine with knowing when care is needed?