Overview
Mental-health conversational AI can sound warm and safe while disrupting therapy or mishandling acute risk. This series separates therapeutic fidelity from crisis safety, then adds context, severity, multi-turn, factuality, demographic, and red-team tests. It shows how synthetic and LLM-judged evidence can expose failure mechanisms and guide mitigations, while remaining bounded: benchmark results do not prove safe real-world outcomes. The practical goal is a traceable evidence package for limited, human-accountable deployment claims.
Learning outcomes
- Warmth, readability, and refusal rates cannot substitute for clinical evaluation.
- Therapeutic fidelity and crisis safety are separate obligations requiring context-specific tests.
- Severity, prior turns, persistence, and subgroup or language variation can expose failures hidden by averages.
- Synthetic and LLM-judged tests can expose failure mechanisms but do not prove safe real-world outcomes.
- Benchmark results support bounded claims; human oversight and ongoing monitoring remain necessary.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
Why test therapeutic fidelity and crisis safety separately?
They measure different clinical obligations: fidelity asks whether the system preserves a therapy protocol, while crisis safety asks whether it recognizes and responds to acute risk. A model can pass one and fail the other.
What should a mental-health AI evaluation measure?
It should combine protocol-specific fidelity, focused crisis safety, hallucination, cross-turn consistency, demographic robustness, and multi-turn stress tests. The portfolio should match the system's intended use and preserve relevant context.
Why do severity and multi-turn testing matter?
They show whether performance changes as risk rises or users persist, exposing failures that single-turn tests and pooled averages can hide. High-risk results should retain their denominators and uncertainty.
Do these benchmarks prove safe clinical deployment?
No. They provide bounded evidence about tested scenarios, not proof of safe outcomes with real patients; human oversight, clinical validation, and ongoing monitoring remain necessary.