The Crisis Cliff: When Safety Behavior Breaks the Therapy
This chapter interprets the supplied PE and CBT evaluations of mental-health conversational AI to ask a narrow but consequential question: what happens when a model's general safety behavior meets a therapy protocol and the apparent severity of the situation rises? The answer is not captured by one average score. Readers need to distinguish surface acknowledgment from therapeutic appropriateness, read high-risk results with their denominators, and identify the different ways a model can interrupt, misunderstand, or abandon clinical work.
Performance does not decline in a straight line
The PE evaluation tested 250 therapy turns drawn from a synthetic trauma-conversation dataset. The turns were classified into four severity levels: routine, distress, crisis-adjacent, and imminent risk. They were not evenly distributed: there were 12 routine turns, 57 distress turns, 175 crisis-adjacent turns, and only 6 imminent-risk turns. The authors say this distribution followed the dataset's classification rather than a balanced design. Four candidate models were assessed by three judges from different model families, producing about 3,000 PE judgments across six dimensions, including acknowledgment, crisis behavior, therapeutic appropriateness, and protocol fidelity.
That design has an important implication. The six imminent-risk turns are too few to support a stable population estimate or a universal ranking of models. They are not a reason to discard the highest-risk result, though. Rare cases can still reveal a deployment-critical failure, especially when an evaluation averages them away. A system that behaves well in hundreds of routine conversations but fails when the user is most vulnerable may have a more serious problem than a system with a lower average but no comparable high-severity collapse.
The results show this difference between ordinary performance and high-stakes performance. At routine severity, Sonnet's mean therapeutic-appropriateness score was 0.94, Qwen's was 0.78, Gemini's was 0.83, and GPT-OSS-20B's was 0.75. At imminent risk, those values fell to 0.72, 0.33, 0.33, and 0.22 respectively. Protocol fidelity fell even more sharply: Sonnet moved from 0.78 at routine severity to 0.11 at imminent risk; Qwen and Gemini reached zero; GPT-OSS-20B reached 0.06. Acknowledgment was much more resilient. Three models remained between 0.89 and 1.00 at imminent risk, although GPT-OSS-20B fell to 0.61.
Those numbers describe a widening gap. A model can continue to recognize that a person is distressed while losing track of what a protocol requires. Acknowledgment is visible in a sentence such as “I hear how overwhelming this is.” Therapeutic appropriateness asks what should happen next within the clinical frame. Protocol fidelity asks whether the response preserves the current phase and task. The first behavior is easier to produce and easier to rate. It is not a substitute for the latter two.
The authors call the non-linear pattern a “crisis cliff.” The term refers to the collapse in model performance as scenarios reach imminent-risk severity, not to a claim that the simulated clients themselves experienced a measured clinical deterioration. Performance declines modestly across lower levels, then drops sharply in the highest-risk cell. GPT-OSS-20B also produced 20 empty or near-empty responses across the PE evaluation, 16 of them at crisis-adjacent severity. That is a different route to failure from an overprotective response: the system may become silent or incomplete precisely when a user needs a coherent interaction.
The point of the state diagram is not to predict a model's next answer. It is to show why severity must be treated as an evaluation dimension. If routine and imminent-risk turns are pooled, the abundant lower-severity cases can dominate the average. The resulting number may describe the common case while hiding the dangerous transition.
This is also why the phrase “the model is safe” is too coarse. A model may be safe for a limited, low-risk informational task and unsafe for autonomous crisis decisions. It may preserve a therapy task at routine severity and lose it when distress language appears. The evidence supports measuring the transition, not assuming that capability at one point carries across the whole severity range.
The CBT evaluation provides a second view of that transition. It used 146 therapist-curated cognitive-restructuring exercises and added 29 severity-escalated variants by placing distress or self-harm language before otherwise benign tasks. The underlying restructuring task remained in place. In the base set, overall scores ranged from 0.93 to 0.96. Under escalation, GPT-OSS-20B fell from 0.93 to 0.74, with clinical accuracy dropping from 0.97 to 0.72 and completeness from 0.92 to 0.71. Sonnet fell from 0.96 to 0.84, while Qwen and Gemini changed by less than 0.01 overall in this set.
The CBT result is not a perfect mirror of PE. The CaiTI exercises were therapist-curated but not anchored to one manualized treatment protocol, so the study provides a weaker basis for a universal CBT fidelity threshold. It does, however, strengthen the narrower interpretation that the problem is not unique to trauma exposure. When the task is cognitive restructuring, a model can respond to risk language by wrapping the exercise in safety content or by failing to complete it. The form changes; the pressure-sensitive loss of clinical function remains visible.
Four numbers do not tell the whole failure story
The aggregate drop matters, but the mechanism matters more. The PE study identifies three patterns that a generic safety test might reward instead of penalize.
The first is premature grounding. During imaginal exposure, models directed the client toward the present room, their breathing, or the feeling of their feet on the floor. In an ordinary support conversation, that can look like a sensible attempt to reduce distress. In active exposure, it can interrupt the required engagement with the memory. The authors counted 162 such flagged cases for Qwen, 173 for GPT-OSS-20B, and 139 for Gemini. The response is not simply “too cautious.” It has substituted a different intervention for the one being evaluated.
A representative scenario involved a client describing a spinning car, blinding headlights, and intense shaking while explicitly identified as being midway through imaginal exposure. The protocol-following reference asked for a distress rating, maintaining contact with the memory. A model instead reassured the client that they were safe in the room and suggested a slow breath. The words are warm. The clinical question is whether they preserve the exposure. Under that question, the reassurance and grounding are a failure path.
The second pattern is memory–reality confusion. Gemini showed 49 identified cases in which it treated a past event being recalled in therapy as if it were happening in real time. In one example, the previous turns established that a client was recounting a hostage situation during exposure. The model nevertheless told the client to follow authorities, move to a secure place, and respond to the emergency scene. This is not merely an unnecessary crisis referral. It is a failure to represent the conversational state: a memory narrative has been mistaken for a current event.
This distinction is crucial for testing. A keyword-based system may see “gunman,” “sirens,” or “weapon” and trigger an emergency script. A context-sensitive system must also ask whether the conversation establishes that the danger is present now. A benchmark that removes the preceding turns may make the wrong response look reasonable, because it has removed the evidence needed to interpret the words.
The third pattern is crisis-resource insertion. Across 274 flagged cases, models inserted hotline references or emergency-service recommendations into controlled exercises where the paper's rubric judged them clinically inappropriate. Gemini accounted for 139 cases, Qwen for 83, GPT-OSS-20B for 50, and Sonnet for 2. The problem was not the existence of crisis resources or the general value of human support. It was the model's failure to distinguish a therapy exercise from an acute crisis.
The authors also report that 34.4% of Sonnet responses, 36.8% of Qwen responses, and 41.6% of Gemini responses contained the phrase “you are safe,” compared with 1.6% of GPT-OSS-20B responses. In active PE, the paper treats that reassurance as potentially counter-therapeutic because it supplies external soothing where the client is meant to remain engaged with the memory. The comparison should not be read as a universal ban on the phrase. It shows how a sentence that sounds benign in general dialogue becomes a protocol marker in a particular phase.
These patterns share a plausible explanation: general safety behavior is being applied without enough clinical context. The study describes models as optimized for broad helpfulness, agreeableness, and harm avoidance. That is an interpretation of the observed outputs, not a controlled experiment isolating one training stage. The evidence can therefore support the claim that safety-aligned behavior and therapeutic fidelity came into conflict in the tested settings; it cannot prove that RLHF alone caused every failure.
The CBT results show two additional forms of disruption. Sonnet's safety-interference score fell from 0.99 in the base set to 0.61 under escalation, and the rate at which a judge detected a safety insertion rose from 0.7% to 31%. Its recurring pattern was to acknowledge the self-harm language, complete the cognitive restructuring, and append a safety check. That response may look like a reasonable compromise, but the study interprets the added safety scaffolding as interference with the exercise. GPT-OSS-20B showed a quieter failure: completeness fell from 0.92 to 0.71, Stage 3 was sometimes omitted, responses were truncated, and two cases were empty. Its safety-triggered rate remained at zero because it did not visibly insert safety content; it simply failed to do the clinical work.
This contrast matters for evaluation design. Visible safety language is not the only failure mode. A test that looks only for harmful advice or explicit crisis escalation could miss omission, truncation, or task abandonment. Conversely, a test that counts every safety mention as good behavior could reward the intervention that breaks the treatment. The evaluator needs separate labels for what the model said, what it failed to do, and whether the clinical context made that behavior appropriate.
The judging process adds another layer of uncertainty. Across 2,993 PE judgments, the three-judge panel achieved a Fleiss' kappa of 0.511 and 73.7% unanimous agreement. Agreement was high for acknowledgment, with κ = 0.872, but much lower for therapeutic appropriateness, with κ = 0.272. The “no false reassurance” dimension did not achieve reliable measurement in this iteration, with κ = −0.001. These values do not invalidate the directional patterns, but they warn against treating every individual score as precise.
The judges also differed systematically. Opus 4.6 gave therapeutic appropriateness a mean score of 0.52, while GPT-5.4 gave 0.90. In 396 cases, Opus flagged a response that GPT-5.4 approved. The study's qualitative analysis attributes the difference to calibration: Opus more often recognized the exposure context and penalized protocol violations, while GPT-5.4 evaluated the response as if it were a general supportive conversation and rewarded warmth and safety behaviors. The disagreement is therefore informative. It reveals that the rubric's hardest construct is not a simple property that any capable language model will rate in the same way.
CBT agreement was higher overall, at κ = 0.704, with strong agreement on clinical accuracy and completeness. That difference fits the tasks: structured exercises with explicit stages can be easier to judge than open-ended therapy responses. It does not show that CBT evaluation is solved. It shows that construct clarity and task structure affect measurement reliability.
The PE study used only LLM judges and did not include human clinical adjudication. Its authors state that a clinician panel would be needed before the framework could serve as a formal evaluation standard. The most defensible conclusion is therefore bounded: the experiments identify a severity-sensitive, cross-modality pattern and several concrete failure mechanisms. They do not establish a population failure rate, prove a single causal training explanation, or certify any model for clinical use.
That boundary is not a weakness to hide. It is part of what a useful evaluation should reveal. Once the model's surface warmth, clinical task, crisis context, and judging uncertainty are separated, the next question becomes clearer: how can a dedicated safety test measure acute risk without pretending to measure the therapy itself?