Curbside Chatbots: Auditing the Mental Health Interventions Large Language Models Recommend to Clinicians, Caregivers, and Patients
Lists
Only you see this
Abstract
Users are asking language models mental health questions and are receiving answers. We audited what 41 language models recommend to users when the same clinical information reaches them from three footings: a clinician describing a patient, a caregiver describing a loved one, or a person describing themselves. Across 19 case vignettes and five questions, we collected 35,055 free-text responses, extracted every intervention named with a validated judge, and had 29 models rate the full option lists on a seven-point Likert scale for the same cases. Models largely recommend what they rate well, with recommendation behavior differing by footing, and many models offering clinicians a consultation-style response and lay users hedged, simplified advice. When asked about recommendations for a specific intervention type, models more often named options they themselves rated below zero as initial choices, usually with conditions attached. We will release the instrument, code, and full corpus on publication as a living audit.
BibTeX
@misc{flathers2026curbside,
author = {Matthew Flathers and Jill Noorily and Phuong Anh Nguyen and Mark Kalinich and Mina Fallah and Victoria Lisowski and Christine Shi and Rebekah M. Bodner and Anna Simms and Mihika Hete and Meiting Chen and Samuel K. Powell and Tatum Williamson and Laura Ospina-Pinillos and Haruka Notsu and Julian Herpertz and John Torous},
title = {{Curbside Chatbots: Auditing the Mental Health Interventions Large Language Models Recommend to Clinicians, Caregivers, and Patients}},
year = {2026},
eprint = {2611.12814},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.12814},
}Loading related content...