Testing Crisis Safety on Its Own Terms
This chapter examines VERA-MH, a focused evaluation for the crisis-safety behavior of mental-health chatbots. Its value lies in the boundary it draws: the test asks whether a system recognizes and responds to suicidal-ideation risk, not whether it delivers effective psychotherapy. That narrow claim makes the evaluation easier to inspect and calibrate, while also showing why it must sit beside a separate test for therapeutic fidelity.
A narrow question with a full conversation behind it
Suicidal ideation, often shortened to SI, refers here to thoughts or expressions related to suicide. VERA-MH does not try to turn every mental-health risk into one universal score. Its first iteration focuses on how a chatbot responds when a user might be in crisis. The designers argue that a narrow scope improves construct validity: the degree to which a measurement actually represents the property it claims to measure.
That choice matters because “mental-health safety” contains many different questions. Can a system recognize risk? Can it clarify whether the risk is present and immediate? Can it guide someone toward human care when indicated? Can it remain supportive without encouraging dependence on the AI? Can it respect its own limits? Those questions are related, but they do not answer whether a model followed a Prolonged Exposure phase or completed a CBT cognitive-restructuring task. VERA-MH is designed for the crisis branch of the problem, not for treatment efficacy or protocol fidelity.
The evaluation has three connected parts. First, a supporting language model plays the role of a user whose behavior is specified by a clinical and demographic persona. That simulated user interacts with the chatbot under evaluation over multiple turns. Second, another model judges the resulting conversation using a clinically developed rubric. Third, the individual judgments are aggregated into a rating for the evaluated system.
The order creates an important division of responsibility. The persona and user-simulation model provide variation in how risk may be expressed. The chatbot produces the behavior being assessed. The judge applies the clinical expectations. The rating stage summarizes the judged conversations. No single component is the whole evaluation, and each component introduces assumptions that need to remain visible.
VERA-MH uses 100 personas with both demographic and clinical characteristics. The clinical dimensions include risk level, history of suicidal thoughts, disclosure style, access to mental-health care, social isolation, and recent triggers. The reported design allocates 30% of personas to high risk, 30% to medium risk, 30% to low risk, and 10% to a no-risk control group. This is a sampling design for evaluation, not a claim about how risk is distributed in the real population.
The persona design also illustrates why realism cannot be assumed from diversity labels. The authors do not include race as a category and do not pass names to the system prompt, in part to reduce stereotyping and identity bias. They still recognize that a language model may stereotype people at the intersection of multiple identities. The personas primarily reflect a US-based population and value system, so applying the evaluation elsewhere requires adaptation rather than a silent assumption that the same representations fit every community.
The conversations are dynamic rather than scripted. A fresh user-model instance generates each conversation, and users may end an interaction when they no longer expect useful information.