Detecting mental disorder on social media: A ChatGPT-augmented explainable approach

Loris Belcastro, Riccardo Cantini, Fabrizio Marozzo, Domenico Talia, Paolo Trunfio

TL;DR

This paper introduces BERT-XDD, a novel self-explainable AI system that accurately detects depression from social media posts. It combines a specialized BERT model (BERTweet) with an attention-based LSTM for classification and explanation, and then uses ChatGPT to translate these technical explanations into human-readable summaries, making it a valuable tool for mental health professionals.

What Problem Does This Paper Address?

The increasing prevalence of mental health issues, particularly depression, coupled with the vast amount of personal data shared on social media, presents a critical opportunity for early detection. However, existing AI models for depression detection often lack interpretability, meaning users (especially clinicians) can't understand why a model makes a certain prediction. This lack of transparency hinders trust and adoption in sensitive domains like mental healthcare. The paper aims to bridge this gap by developing a system that not only accurately detects depression but also provides clear, understandable explanations for its predictions.

Methods

The core of the proposed methodology is the BERT-XDD model, which is built in two main stages:

  1. Pre-fine-tuning BERTweet: BERTweet, a BERT variant specifically trained on Twitter data, is fine-tuned for depression classification. This stage leverages transfer learning to adapt the model to the specific task.
  2. BERT-XDD Model Training: The fine-tuned BERTweet encoder is integrated with an attention-based bi-directional LSTM. This LSTM is designed to generate explanations by assigning weights to individual words in a post, indicating their importance for the classification. This is achieved through a masked attention mechanism that focuses on relevant words and filters out noise.

To enhance interpretability further, ChatGPT is employed. The technical explanations (word weights) generated by BERT-XDD are fed to ChatGPT along with the original post and the predicted class. Using carefully engineered prompts (including a one-shot learning example), ChatGPT transforms these weights into concise, human-readable commentaries that explain the model's decision.

Datasets

The methodology was evaluated on two datasets:

  • Twitter Dataset: This dataset, adapted from a competition (LT-EDI-ACL2022), contains 10,251 tweets labeled into three categories: SEVERELY_DEPRESSED, MODERATELY_DEPRESSED, and NOT_DEPRESSED.
  • Reddit Dataset: This dataset comprises 7,650 posts from Reddit, annotated as either DEPRESSED or NOT_DEPRESSED.

Key Insights and Ideas

  • Bridging the Accuracy-Interpretability Gap: The paper demonstrates that it's possible to achieve high classification accuracy while also providing meaningful explanations, addressing a common trade-off in AI for sensitive applications.
  • ChatGPT as an Explanation Translator: Leveraging ChatGPT for translating technical AI explanations into natural language is a powerful way to make complex models accessible to non-expert users, like healthcare professionals.
  • Importance of Prompt Engineering: The study highlights that the effectiveness of ChatGPT in generating useful explanations is highly dependent on the design of the prompts, with advanced prompts leading to more concise and focused outputs.
  • BERTweet's Suitability for Social Media: The research confirms BERTweet's effectiveness in handling the nuances of social media text (hashtags, mentions, emojis) for depression detection.

Key Results

  • Superior Classification Performance: BERT-XDD achieved the best results across both datasets, outperforming state-of-the-art models like DepRoBERTa and its ensemble version. For instance, on the Reddit dataset, BERT-XDD achieved an accuracy of 0.991 and a Macro-F1 score of 0.991.
  • Effective Explanations: The ChatGPT-augmented explanations were shown to be clear and insightful. The advanced prompt consistently produced more concise and focused commentaries that accurately reflected the BERT-XDD model's reasoning, avoiding extraneous information.
  • Model Comparison: BERTweet was found to outperform other BERT-like models (BERT, RoBERTa, ALBERT) in the pre-fine-tuning step, making it a strong choice for social media text analysis.

Limitations and Future Work

  • Static Analysis: The current methodology analyzes individual posts. Future work could explore the dynamic nature of mental states by analyzing entire conversational threads.
  • Real-time Application: The authors suggest adapting the methodology for real-time analysis to enable timely alerts to healthcare professionals, facilitating early intervention.
  • Broader LLM and XAI Integration: The modular nature of the approach allows for integration with various LLMs and XAI techniques, which could be explored further.
  • Target Audience: The methodology is intended for use by qualified healthcare professionals, who can validate and leverage the results in their clinical assessments.

Summary updated

Abstract

In the digital era, the prevalence of depressive symptoms expressed on social media has raised serious concerns, necessitating advanced methodologies for timely detection. This paper addresses the challenge of interpretable depression detection by proposing a novel methodology that effectively combines Large Language Models (LLMs) with eXplainable Artificial Intelligence (XAI) and conversational agents like ChatGPT. In our methodology, explanations are achieved by integrating BERTweet, a Twitter-specific variant of BERT, into a novel self-explanatory model, namely BERT-XDD, capable of providing both classification and explanations via masked attention. The interpretability is further enhanced using ChatGPT to transform technical explanations into human-readable commentaries. By introducing an effective and modular approach for interpretable depression detection, our methodology can contribute to the development of socially responsible digital platforms, fostering early intervention and support for mental health challenges under the guidance of qualified healthcare professionals.
BibTeX
@article{belcastro2025detecting,
  author = {Loris Belcastro and Riccardo Cantini and Fabrizio Marozzo and Domenico Talia and Paolo Trunfio},
  title = {{Detecting mental disorder on social media: A ChatGPT-augmented explainable approach}},
  journal = {Online Soc. Networks Media},
  year = {2025},
  doi = {10.1016/j.osnem.2025.100321},
}
Figure