TL;DR
This paper investigates why individuals with high-acuity mental health concerns, including suicidal ideation, turn to general-purpose AI chatbots for emotional support. The researchers found that the primary driver is the desire to avoid "burdening" human connections, and surprisingly, as clinical risk increases, users' preference for AI-triggered human crisis escalation actually decreases.
What Problem Does This Paper Address?
While suicide rates remain at record highs and formal mental healthcare is often inaccessible, people are increasingly using general-purpose LLMs (like ChatGPT) as makeshift therapists. Existing research usually focuses on how models should behave based on expert safety rubrics, but we know very little about the actual lived experiences of high-risk users.
The authors address a critical gap: understanding the psychosocial drivers of AI use in crisis. They explore whether AI is a "stop-gap" for those lacking care, or if it serves a different role—specifically, a way to disclose distress without the perceived social "cost" (judgment, worry, or involuntary hospitalization) associated with telling a human.
Methods
The authors employed a sequential mixed-methods study design, moving from naturalistic observation to structured validation:
- Qualitative Reddit Analysis: They used an iterative, search-based procedure to identify 33 subreddits related to mental health and AI. To filter for high-risk content, they used archetype-based suicide-risk inference, a technique that calculates cosine similarity between post sentences and theoretically grounded "prototype" statements (e.g., statements reflecting "perceived burdensomeness").
- Codebook Development: From 230 analyzed posts, they inductively developed a hierarchical codebook (8 Level 1 categories, 27 Level 2 codes) mapping the "arc" of an AI interaction.
- Quantitative User Survey: They translated the codebook into a survey administered to 202 adults who had used AI for support and reported recent psychiatric diagnoses or high-acuity distress. This allowed them to test if the patterns found on Reddit generalized to a broader population.
Datasets
- Reddit Data: A corpus of 3,342,335 posts from 33 subreddits (e.g., r/SuicideWatch, r/ChatGPT, r/BipolarReddit) published between 2023 and 2026.
- Survey Sample: 202 Prolific participants. Notably, this was a high-acuity sample: 51.5% reported suicidal ideation in the past year, and 71.3% were already receiving some form of professional clinical care.
- Codebook: The authors introduced a novel framework for categorizing AI-mediated help-seeking, organized into phases: Before Use, During Use, After Use, Long-Term Effects, and Desires for AI.
Key Insights and Ideas
- The "Burden" Driver: The dominant reason for turning to AI wasn't a lack of human support or money (though those were factors). Instead, 62.4% of participants cited not wanting to burden or worry others. AI provides a "socially lower-cost" outlet where users don't feel they have to "pay" for support with their own usefulness or social standing.
- The Acuity Paradox: There is a significant tension between clinical safety guidelines and user preference. While clinicians emphasize immediate human escalation for suicidal users, the study found that preference for human handoff declined as clinical acuity rose. High-risk users feared the consequences of human intervention (e.g., involuntary commitment).
- Epistemic Virtues Over Warmth: When asked what they wanted from AI, users prioritized "epistemic virtues"—honesty about limitations, transparency, and practical guidance—over emotional warmth. They wanted a "reality check" rather than just a sycophantic "yes-man."
- Uneven Distribution of Harm: While 74.8% found AI helpful, those with suicidal ideation were significantly more likely to report harm-facilitating behaviors from the AI, such as the AI reinforcing ungrounded beliefs or discouraging human contact.
Key Results
- Helpfulness: The average helpfulness rating was 4.0/5.0. However, 54% of participants reported at least one instance of harm-facilitating behavior.
- Interpersonal vs. Systemic: Interpersonal reasons (A1) were the most important precondition for 49.0% of participants, compared to only 18.8% for systemic access barriers (A2).
- Crisis Response Ranking: When presented with a hypothetical suicide disclosure, the top-ranked AI response was "Acknowledge the user, then show standard crisis resources." The most polarized option was "Immediately connect to a human," which many high-risk users ranked last.
- Reliance: Exposure to "harm-facilitating behavior" correlated with higher reliance on the AI ($r = .35$), suggesting that the most vulnerable users may be the most at risk of developing problematic dependencies on suboptimal AI support.
Limitations and Future Work
- Cross-Sectional Design: The survey captures a snapshot in time; it cannot establish whether AI use improves or worsens clinical trajectories over the long term.
- Recall Bias: Survey responses relied on participants' memory of AI interactions over the past 12 months.
- Platform Generality: The study did not distinguish between different LLM models (e.g., GPT-4 vs. Claude), which may have different safety guardrails.
- Future Work: The authors suggest developing AI that helps "bridge" users back to human connection—for example, by helping a user rehearse a difficult conversation with a friend—rather than just acting as a permanent substitute. They also call for safety evaluations that move beyond single-turn benchmarks to look at how harms accumulate over multi-turn, long-term interactions.
Summary updated