Are You Really Okay? A Transfer Learning-based Approach for Identification of Underlying Mental Illnesses

Ankit Aich, Natalie PardeNAACL 2022

TL;DR

This paper introduces a novel transfer learning approach to identify underlying mental health conditions (MHCs) by leveraging linguistic patterns learned from known conditions. By training models on datasets for depression, stress, and suicide, they demonstrate that these learned patterns can be effectively transferred to predict other, potentially latent, MHCs, achieving state-of-the-art performance.

What Problem Does This Paper Address?

Mental health conditions are a significant societal concern, and while automated detection using language analysis has advanced, most research focuses on individual conditions in isolation. This paper addresses the gap in understanding and leveraging the interplay and overlap between different MHCs. The authors argue that these shared linguistic characteristics, often overlooked, represent a valuable source of information, particularly in resource-limited settings. They aim to explore how linguistic features relate across MHCs, whether different MHCs can be represented in shared feature spaces, and ultimately, if language models can identify underlying MHCs by learning from known ones.

Methods

The core of this work is a transfer learning framework. The authors propose a model that learns linguistic feature representations from a "source" MHC and applies them to predict a "target" MHC. They explore two main conditions:

  1. Same Feature Space: Using LIWC (Linguistic Inquiry and Word Count) features for both source and target tasks. LIWC is a well-established tool that extracts linguistic features related to psychological meaning.
  2. Different Feature Spaces: Using LIWC features for the source task and Word2Vec embeddings for the target task. Word2Vec captures semantic relationships between words based on their context.

The models used include Convolutional Neural Networks (CNNs) and Bidirectional Long Short-Term Memory (BiLSTM) networks. The transfer learning process involves training on a source task, freezing the dense layers of the model, and then training on a target task. They also employ Latent Dirichlet Allocation (LDA) for topic modeling and analyze n-grams to understand thematic overlaps.

Datasets

The study utilizes publicly available datasets for three mental health conditions:

  • Depression: Sourced from Twitter, with binary labels (depression/no depression).
  • Stress: The "Dreaddit" dataset, sourced from Reddit, with binary labels (stressed/not stressed).
  • Suicide: Sourced from Reddit, with categorical labels (None, Low, Moderate, Severe) which were aggregated into binary labels for some analyses.

The authors emphasize the challenge of acquiring such datasets due to IRB restrictions and user privacy concerns.

Key Insights and Ideas

  • Shared Linguistic Signatures: Mental health conditions, despite their differences, exhibit significant overlap in their linguistic manifestations. This suggests that language can serve as a common thread for understanding and potentially identifying various MHCs.
  • LIWC's Advantage: Features that explicitly encode psychological meaning (like LIWC) are more effective for transfer learning across MHCs than purely distributional semantic features (like Word2Vec). This highlights the importance of linguistically grounded features in this domain.
  • Psychologically Informed Transfer: The order of transfer learning matters. A model trained on stress, then depression, and finally suicide achieved higher accuracy for suicide prediction, aligning with the clinical understanding of suicide as a potential escalation from stress and depression.
  • AI as a Tool, Not a Replacement: While AI models can achieve impressive accuracy in identifying potential MHCs, they are not a substitute for professional mental health care and cannot conclusively diagnose illnesses.

Key Results

The proposed transfer learning model achieved strong performance, outperforming existing benchmarks:

  • Depression: F₁ score of 0.75 (13.64% improvement).
  • Stress: F₁ score of 0.80 (1.27% improvement).
  • Suicide: F₁ score of 0.76 (80.95% improvement).

The model achieved a new state-of-the-art performance for these tasks. Furthermore, a psychologically informed transfer learning approach (stress → depression → suicide) achieved 85% accuracy in predicting suicidal ideation.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Generalizability: Models trained on social media data may not always generalize well to other contexts, and performance can be impacted by factors like social concerns, self-disclosure bias, and temporal artifacts.
  • Feature Space Exploration: While LIWC proved superior, further exploration of different feature spaces and their combinations is warranted.
  • Model Variety: The study focused on CNNs and BiLSTMs; future work could explore a wider range of models.
  • Transfer Learning Order: The impact of transfer learning order on performance needs further investigation.
  • Broader MHC Coverage: Expanding the approach to include a larger variety of mental health conditions would be beneficial.
  • Ethical Considerations: Continued focus on ethical data handling and responsible AI deployment in mental health is crucial.

Summary updated

BibTeX
@inproceedings{aich2022are,
  author = {Ankit Aich and Natalie Parde},
  title = {{Are You Really Okay? A Transfer Learning-based Approach for Identification of Underlying Mental Illnesses}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure