Crisp: Cognitive Restructuring of Negative Thoughts through Multi-turn Supportive Dialogues

Jinfeng Zhou, Yuxuan Chen, Jianing Yin, Yongkang Huang, Yihan Shi, Xikun Zhang, Libiao Peng, Rongsheng Zhang, Tangjie Lv, Zhipeng Hu, Hongning Wang, Minlie HuangarXiv 2025 · DOI · OpenReview

TL;DR

This paper introduces CRDIAL, a novel framework for training AI to perform Cognitive Restructuring (CR) through multi-turn dialogues. They've created CRISP, a large, high-quality bilingual dataset for this task, and trained CRISPERS, LLMs that demonstrate superior performance in improving users' mental states compared to existing models and even human-like chatbots.

What Problem Does This Paper Address?

Mental health challenges often stem from negative thought patterns, and Cognitive Restructuring (CR) is a proven therapeutic technique to address these. However, a global shortage of mental health professionals and the stigma associated with seeking help limit access to CR. While LLMs offer a promising avenue for AI-driven CR, current approaches fall short. They often rely on simplistic text rewriting, rigid dialogue structures, or a single-interaction approach that fails to capture the nuanced, multi-stage process of real-world CR. This paper aims to bridge this gap by developing an LLM-based CR system that mimics the iterative, supportive, and multi-faceted nature of human-led therapy.

Methods

The core of this work is the CRDIAL framework, designed to simulate CR dialogues. It operates in two main stages:

  1. CT-guided Identification Stage: This stage uses Cognitive Theory to guide the LLM therapist in identifying an individual's negative thoughts. It probes three levels: automatic thoughts (surface-level), intermediate beliefs (underlying assumptions), and core beliefs (deep-seated cognitive distortions). This stage is designed to be multi-turn and adaptive, allowing for the exploration of multiple distortions.
  2. DAT-driven Restructuring Stage: This stage employs the Defense Attorney Technique (DAT), using a courtroom analogy. The LLM therapist guides the individual to "defend" their negative thoughts by presenting evidence, then "prosecutes" these thoughts by challenging the evidence and offering counterarguments. This aims to transform distorted thoughts into more positive and helpful ones.

Key innovations within CRDIAL include:

  • Multi-stage, Multi-turn CR: Mimicking the progressive nature of therapy.
  • Sentence-level Supportive Conversation Strategies: Integrating psychological theories (like Hill's Help Skill Theory) to ensure empathetic and effective responses at the sentence level.
  • Multi-channel Loop Mechanism: Allowing the framework to identify and address multiple cognitive distortions within a single mental health issue, accommodating individual differences.

Datasets

The paper introduces CRISP (Cognitive Restructuring Interactive Supportive Process), a novel, large-scale, high-quality bilingual dialogue dataset for CR. It was distilled from GPT-40 using 2,985 seed situations and comprises 22,063 dialogues. CRISP is notable for its multi-channel loop mechanism, leading to an average of 36.48 turns per dialogue and high lexical diversity, making it well-suited for addressing diverse cognitive distortions. The dataset is also fine-grained, annotating sentence-level strategy constraints and cognitive distortion types.

Key Insights and Ideas

  • Mimicking Therapeutic Nuance: The CRDIAL framework's success highlights the importance of replicating the multi-stage, iterative, and emotionally supportive nature of human therapy for effective AI-driven CR. Simple text rewriting or one-shot interactions are insufficient.
  • The Power of Multi-channel Dialogue: The ability to identify and address multiple cognitive distortions simultaneously (the multi-channel loop mechanism) is crucial for handling the complexity of individual mental health challenges.
  • Sentence-Level Strategy Control is Key: Fine-grained control over supportive strategies at the sentence level significantly boosts the quality and effectiveness of AI-generated therapeutic dialogues.
  • LLMs Can Achieve High Performance with Quality Data: The CRISPERS models, trained on the carefully curated CRISP dataset, outperform larger models like GPT-40 on several metrics, demonstrating that data quality can be as important as model size.
  • CRISP is a Valuable Resource: This new dataset offers a significant advancement for CR research, providing a rich, bilingual resource for training and evaluating CR-focused dialogue systems.

Key Results

  • CRISPERS-14B Outperforms Baselines: The trained CRISPERS models, particularly the 14B version, consistently outperformed baseline LLMs (including GPT-40) in pointwise and pairwise evaluations across various mental health situations.
  • Superiority in Key Metrics: CRISPERS-14B achieved significantly higher scores in supportiveness, helpfulness, and trustworthiness, indicating its ability to foster positive thinking and earn user trust.
  • Bilingual Effectiveness: The model demonstrated strong performance in both English and Chinese, highlighting its potential for broader accessibility.
  • Intervention Trial Success: In a psychological intervention trial, CRISPERS-14B significantly improved participants' positive affect and reduced negative affect, outperforming both GPT-40 and a commercial chatbot (Emohaa).
  • Ablation Studies Confirm Component Importance: Removing sentence-level strategy control or multi-channel distortion identification from CRISPERS-14B led to significant drops in performance, underscoring the importance of these novel components.

Limitations and Future Work

  • LLM Choice for Dialogue Construction: The authors used GPT-40 to generate dialogues. Future work could explore other LLMs to diversify dialogue content and style.
  • Alternative Backbone Models: Investigating different backbone models for CRISPERS could lead to variations in response quality.
  • Annotation Bias: Despite rigorous quality control and expert review, the possibility of unintended biases from annotators remains.
  • Intervention Experiment Scale: While promising, the intervention trial was conducted with a limited number of participants. Expanding the scale for future studies is planned.
  • High-Risk Situations: The authors acknowledge that their model is not a substitute for professional therapy, especially in high-risk situations like self-harm or suicide.
  • Exploring Other LLMs: Future research will investigate other potential LLMs for dialogue construction and alternative backbone models for CRISPERS.

Summary updated

Abstract

Cognitive Restructuring (CR) is a psychotherapeutic process aimed at identifying and restructuring an individual's negative thoughts, arising from mental health challenges, into more helpful and positive ones via multi-turn dialogues. Clinician shortage and stigma urge the development of human-LLM interactive psychotherapy for CR. Yet, existing efforts implement CR via simple text rewriting, fixed-pattern dialogues, or a one-shot CR workflow, failing to align with the psychotherapeutic process for effective CR. To address this gap, we propose CRDial, a novel framework for CR, which creates multi-turn dialogues with specifically designed identification and restructuring stages of negative thoughts, integrates sentence-level supportive conversation strategies, and adopts a multi-channel loop mechanism to enable iterative CR. With CRDial, we distill Crisp, a large-scale and high-quality bilingual dialogue dataset, from LLM. We then train Crispers, Crisp-based conversational LLMs for CR, at 7B and 14B scales. Extensive human studies show the superiority of Crispers in pointwise, pairwise, and intervention evaluations.
BibTeX
@misc{zhou2025crisp,
  author = {Jinfeng Zhou and Yuxuan Chen and Jianing Yin and Yongkang Huang and Yihan Shi and Xikun Zhang and Libiao Peng and Rongsheng Zhang and Tangjie Lv and Zhipeng Hu and Hongning Wang and Minlie Huang},
  title = {{Crisp: Cognitive Restructuring of Negative Thoughts through Multi-turn Supportive Dialogues}},
  year = {2025},
  eprint = {2504.17238},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.17238},
}
Figure