What Therapy Is Asking the System to Do
This chapter examines what mental-health conversational AI must preserve when it is placed inside a structured therapy task. The focus is therapeutic fidelity: whether a system follows the purpose and sequence of an intervention, rather than merely sounding warm. Understanding that distinction lets readers judge later model failures as failures of clinical function, not as disagreements about style.
A protocol is a mechanism, not a tone
A therapy protocol is more than a collection of kind phrases. It is an organized account of what the therapist and client are doing, in what order, and why that sequence is expected to help. The order matters because each step creates the conditions for the next one. A system that skips a required step may still produce a compassionate sentence, but it has changed the intervention.
This is what makes therapeutic fidelity a useful term. Fidelity means adherence to the clinically specified purpose, phase, and boundaries of a treatment. It does not require a chatbot to imitate a particular therapist's personality, reproduce one exact sentence, or sound formal. It asks a narrower and more consequential question: did the response do the kind of work that this intervention required at this moment?
That question is different from efficacy. A fidelity test can tell us whether a response followed a protocol marker; it cannot by itself prove that the client improved. Conversely, a short-term improvement would not prove that every response during treatment was faithful or safe. Keeping process and outcome separate prevents a common mistake in AI evaluation: treating a pleasant interaction, or even a promising outcome, as evidence that the system performed the intended clinical mechanism.
Prolonged Exposure therapy for post-traumatic stress makes the issue visible because its active work can look counterintuitive from outside a treatment setting. The supplied study describes three broad phases: therapist orientation, imaginal exposure, and post-imaginal processing. During imaginal exposure, a client deliberately engages with a distressing memory. The goal is not to make the memory pleasant in that moment. The therapeutic task includes staying with the memory, monitoring distress, and allowing the experience to be processed within a structured setting.
That does not mean distress is ignored or that a client should be pushed through danger without support. The protocol has a clinical frame, and the response must remain sensitive to the person's state. The narrower point is that visible distress is not, by itself, evidence that the treatment has gone wrong. In an exposure exercise, interrupting the memory at the first sign of distress can defeat the very engagement the exercise is designed to support.
This creates a sharp test for a conversational system. A general-purpose assistant may have learned that the safest response to distress is to redirect attention to the present, recommend slow breathing, reassure the person that they are safe, or offer crisis resources. Those behaviors can be appropriate in an acute crisis. During active imaginal exposure, however, they may pull the client out of the memory and reinforce avoidance. A protocol-following response might instead ask for a distress rating or continue the narrative within the therapeutic frame. The difference is not a difference in kindness. It is a difference in mechanism.
CBT cognitive restructuring presents the same principle through a different intervention. The supplied CBT evaluation uses exercises with three linked elements: a client's situation, an automatic or unhelpful thought, and therapist-authored responses that challenge that thought and construct a balanced alternative. An automatic thought is an immediate interpretation of an event, often accepted as true before it is examined. Cognitive restructuring does not ask the system simply to contradict the client or replace the thought with a cheerful slogan. It asks the system to engage the reasoning behind the thought and help formulate a more balanced interpretation.
That sequence gives evaluators concrete things to inspect. Did the response address the stated situation? Did it challenge the unhelpful thought in a way relevant to that situation? Did it produce the alternative thought, rather than stopping after validation? A response can be emotionally respectful and still be incomplete if it performs only the first part—or if it abandons the task altogether because the thought contains distressing language.
The CBT dataset is not tied to one manualized treatment protocol in the same way as the PE evaluation. That makes it a weaker basis for declaring a definitive CBT fidelity standard. It is still useful for showing how a structured cognitive task differs from open-ended emotional support. In both cases, the evaluator needs to know what the system was supposed to do before judging whether the response was helpful.
The sequence is intentionally simple. The system cannot be judged from the client statement alone. The same words acquire different significance after the clinical context identifies an exposure exercise, a cognitive-restructuring task, or an acute-risk situation. A fidelity evaluator therefore needs the context and the expected action, not just the generated sentence.
The same safety cue changes meaning inside the task
The most important boundary is that grounding, reassurance, crisis referral, and refusal are not intrinsically correct or incorrect. Their appropriateness depends on present danger, therapy phase, and the task the system has been authorized to perform.
Take reassurance. Telling a person that they are safe can be a useful response when the immediate problem is a present threat and the statement is accurate. But in active exposure, the phrase can shift the focus from processing a traumatic memory to receiving external soothing. The supplied PE study treats this as a potential protocol violation because the client is meant to learn through the exercise that distress can be experienced and processed without escaping the memory at once. The phrase is not dangerous in every conversation; its meaning changes with the intervention.
Grounding illustrates the same boundary. In a crisis or an overwhelming present-moment state, directing attention to surroundings may help stabilize the interaction. In a controlled exposure exercise, directing attention away from the memory may function as avoidance. A general safety benchmark that rewards any grounding response would therefore mistake a context-sensitive intervention for a universal rule.
Crisis resources require an even more careful distinction. When a person expresses imminent danger, connecting them with emergency services or other human support may be necessary. When a client is recounting a past assault as part of a clearly established therapy exercise, inserting a hotline or emergency instruction can wrongly treat the memory as a live event. The error is not that crisis resources exist. The error is that the system has failed to determine which situation it is in.
The boundary also runs in the other direction. It would be wrong to use the conflict between exposure and grounding as an argument against crisis response. A model that remains inside a therapy frame when a client is actually at imminent risk could fail in a more direct way. The evaluator must test whether the system can distinguish a controlled therapeutic exercise from current danger, and whether it can change course when the risk signal warrants it. Clinical appropriateness is conditional rather than a fixed list of preferred phrases.
This conditionality explains why general safety alignment can interfere with therapeutic work without making safety training worthless. Models are often optimized to be broadly helpful, agreeable, and cautious. Those tendencies are valuable in many settings. But they do not automatically encode the boundaries of every evidence-based treatment. A model may have learned a strong association between distress language and safety intervention, while lacking a sufficiently precise representation of the therapy phase that gives the distress a different function.
For evaluators, the remedy is to test the relationship among three elements: the signal in the user's words, the clinical state established by the conversation, and the action required by the protocol. A distressing sentence alone is not enough. The test must establish whether the person is remembering or currently experiencing an event, whether the system is in orientation, exposure, processing, or cognitive restructuring, and whether there is a separate acute-risk indicator. Without those distinctions, the benchmark rewards a response for reacting to keywords rather than for understanding the clinical situation.
A meaningful fidelity benchmark can therefore be built around phase markers and task completion. In PE, it might ask whether the response maintains the exposure, monitors distress at an appropriate point, and avoids an unnecessary departure from the active phase. In CBT, it might ask whether the system completes the challenge-and-alternative arc and preserves the intended reasoning rather than replacing it with generic encouragement. These tests need not demand one wording. They need to identify the clinically relevant behavior and the boundaries around it.
The strongest test cases would make those boundaries explicit. A benchmark could pair a client recalling an assault during an established exposure exercise with a client describing an assault happening now. The language may overlap, but the expected response should not. The first case tests whether the system stays inside the exposure frame; the second tests whether it recognizes a current safety problem and changes course. A similar pair could place the same self-harm phrase inside a CBT restructuring exercise and inside a conversation with evidence of imminent danger. The benchmark would then test whether the system can preserve the cognitive task when appropriate without allowing the task to obscure a crisis signal.
This paired design does more than make the test harder. It identifies what kind of mistake occurred. If the system grounds both clients, it may be over-triggering a generic safety policy. If it treats both as ordinary therapy, it may be missing acute risk. If it completes neither CBT stage after seeing distress language, it may be abandoning the task rather than inserting a visible safety intervention. The response can then be evaluated against the relevant clinical boundary instead of being assigned a vague label such as “helpful” or “unsafe.”
A protocol-aware evaluation should also distinguish a missing action from a harmful action. Failing to ask for a distress rating during exposure is not identical to inserting a crisis hotline, even if both reduce fidelity. Omitting the balanced alternative in CBT is different from challenging the thought in a way that is inaccurate or invalidating. These distinctions matter for remediation: the first may require better task tracking, the second better context recognition, and the third stronger clinical content review. A single overall score would conceal those different routes to failure.
There is also an evidence trade-off between the two therapy examples. The PE evaluation is anchored to a named treatment guide and explicit phase progression, which makes protocol judgments more concrete. The CBT exercise set is carefully curated but not tied to one manual, so it can show task disruption and incomplete cognitive work more readily than it can establish a universal CBT fidelity threshold. A responsible evaluation carries that difference forward instead of presenting both forms of evidence as equally precise.
There is also a trade-off between fidelity and conversational naturalness. A response that strictly follows a protocol may sound less spontaneously comforting than a general-purpose assistant. That does not automatically make it better or worse. Naturalness is valuable when it helps the client engage, but it cannot be the final criterion if it hides a skipped phase or an interrupted mechanism. The evaluator should ask whether stylistic variation preserves the therapeutic function.
The reverse trade-off matters too. A system may follow a protocol marker while communicating in a cold, confusing, or invalidating way. Fidelity is not a license to disregard the person. The first study's proposed rubric separates acknowledgment and other safety behaviors from therapeutic appropriateness and protocol fidelity for precisely this reason. A response can satisfy one dimension and fail another. The goal is not to choose warmth over structure, or structure over humanity, but to measure both without pretending they are interchangeable.
That separation gives the next empirical question its shape. If a model is placed in PE and CBT scenarios with increasing severity, does it preserve the clinical task, or does it switch to a generic safety script? And if evaluators disagree, are they disagreeing about the model's behavior, the protocol, or the rubric used to judge it? The answers require more than a list of desirable phrases. They require severity-stratified experiments that can show where the system's clinical behavior changes.