MentalGLM Series: Explainable Large Language Models for Mental Health Analysis on Chinese Social Media

Wei Zhai, Nan Bai, Qing Zhao, Jianqiang Li, Fan Wang, Hongzhi Qi, Meng Jiang, Xiaoqin Wang, Bing Xiang Yang, Guanghui Fu

Wei Zhai, Nan Bai, Qing Zhao, Jianqiang Li, Fan Wang, Hongzhi Qi, Meng Jiang, Xiaoqin Wang, Bing Xiang Yang, Guanghui Fu.
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

Abstract

Mental health analysis on social media is a critical yet challenging task, often hindered by the "black-box" nature of traditional deep learning models. We present the MentalGLM series, the first open-source Chinese Large Language Models (LLMs) specifically fine-tuned for explainable mental health analysis. By leveraging a novel 9K-sample multi-task dataset (C-IMHI) and a two-stage fine-tuning process, MentalGLM achieves state-of-the-art performance while providing expert-aligned reasoning for its psychological assessments.

Introduction

Social media platforms like Weibo have become vital spaces for individuals to express mental health struggles. However, automated analysis in this domain requires more than just high classification accuracy; it requires interpretability to be useful for clinicians. Traditional models like BERT variants are accurate but opaque, while general-purpose LLMs like GPT-4 often struggle with the cultural nuances and slang of Chinese social media.

MentalGLM identifying a cognitive distortion with a detailed explanation.

MentalGLM addresses this gap by providing explanations grounded in psychological frameworks, such as identifying specific cognitive distortions (e.g., "labeling") and explaining the underlying reasoning, as shown in the figure above.

Methodology

The MentalGLM series is built upon the GLM-4-9B architecture using a sophisticated training pipeline designed for domain expertise:

  1. Knowledge Transfer: The model first undergoes training on a translated version of the English "Interpretable Mental Health Instruction" (IMHI) dataset to establish a baseline for mental health reasoning.
  2. Domain Adaptation via C-IMHI: The core contribution is the C-IMHI dataset, containing 9,251 samples focused on Chinese social media and clinical data. It covers three primary tasks: Suicide Risk Detection, Cognitive Distortion Detection, and Cognitive Pathway Extraction.
  3. Teacher-Student Distillation: To ensure high-quality reasoning, human experts developed reasoning paths that were used to prompt GPT-4. These outputs were then manually validated and corrected by experts to create a gold-standard training set.
  4. Parameter-Efficient Fine-Tuning: The 9B parameter models were fine-tuned using Low-Rank Adaptation (LoRA), ensuring efficiency without sacrificing performance.

Key Results

MentalGLM demonstrates superior performance across both social media and clinical contexts. On the suicide risk detection task (SOS-HL-1K), MentalGLM-chat achieved an 85.12% F1-score, significantly outperforming the previous state-of-the-art, Chinese MentalBERT (81.30%).

Performance metrics including correctness, consistency, and human evaluation scores.

As illustrated in the evaluation metrics above, the model maintains high correctness (95-99% agreement) and consistency (F1-scores > 92%) across various datasets. In human evaluations, MentalGLM’s explanations were rated highly for consistency and reliability, often matching or exceeding the performance of much larger models like GPT-4 in specific psychological reasoning tasks.

Furthermore, the model showed remarkable generalization to clinical settings. Despite being trained primarily on social media data, it achieved a 69.19% F1-score on clinical cognitive pathway extraction, nearly 25 percentage points higher than non-fine-tuned models like Llama-3.

Discussion and Conclusion

MentalGLM represents a significant step toward bridging the gap between automated screening and clinical utility. By providing explanations based on Cognitive Behavioral Therapy (CBT) frameworks, it serves as a tool to assist psychologists in identifying maladaptive behaviors.

While the model excels in accuracy and reliability, there is room for improvement in its "Professionality" score—the ability to mimic a clinical expert's tone—which may be limited by the 9B parameter size. Future work will focus on incorporating more de-identified clinical data and addressing potential biases to further enhance the model's utility in hospital settings. MentalGLM is intended as an auxiliary tool for research and analysis, not a replacement for professional medical judgment.

Summary updated

BibTeX
@misc{zhai2024mentalglm,
  author = {Wei Zhai and Nan Bai and Qing Zhao and Jianqiang Li and Fan Wang and Hongzhi Qi and Meng Jiang and Xiaoqin Wang and Bing Xiang Yang and Guanghui Fu},
  title = {{MentalGLM Series: Explainable Large Language Models for Mental Health Analysis on Chinese Social Media}},
  year = {2024},
  eprint = {2410.10323},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.10323},
}
Figure