Deep Learning for Depression Detection of Twitter Users

Ahmed Husseini Orabi, Prasadith Buddhitha, Mahmoud Husseini Orabi, Diana InkpenNAACL 2018

TL;DR

This paper explores using deep learning, specifically Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), to detect depression in Twitter users based on their posts. They introduce an optimized word embedding approach that significantly improves detection accuracy, outperforming traditional methods and even other deep learning architectures.

What Problem Does This Paper Address?

Mental illnesses, particularly depression, are a significant global health concern. Identifying individuals at risk early can lead to timely intervention. Social media platforms like Twitter offer a rich, albeit noisy, source of user-generated content that can reflect mental states. However, detecting mental health issues from this data is challenging due to the unstructured nature of text, the complexity of mental disorders, and the difficulty in obtaining large, labeled datasets for training machine learning models. Existing approaches often rely on extensive manual feature engineering, which can be time-consuming and may not capture the full spectrum of linguistic cues.

Methods

The core of this research lies in applying and optimizing deep learning architectures for depression detection. The authors investigate two main families of models:

  • Convolutional Neural Networks (CNNs): They explore several CNN variants, including CNNWithMax, MultiChannelCNN, and MultiChannelPoolingCNN. These models use convolutional filters to extract local features from text, followed by pooling layers to aggregate information.
  • Recurrent Neural Networks (RNNs): Specifically, they utilize Bidirectional Long Short-Term Memory (BiLSTM) networks with an attention mechanism (BiLSTM (Context-aware attention)). RNNs are well-suited for sequential data like text, capturing temporal dependencies.

A key contribution is their optimized word embedding approach. Instead of using standard pre-trained embeddings (like Word2Vec's Skip-gram or CBOW) or random embeddings, they propose a method that leverages both word prediction and "sense" prediction (classifying the context as depressed, PTSD, or neither). This multi-task learning approach, illustrated in Figure 2, aims to create embeddings that are more attuned to health-specific language. They also experimented with different word embeddings (optimized, trainable, Skip-gram, CBOW) and hyperparameters.

Datasets

The study utilizes two datasets:

  • CLPsych 2015 Shared Task Dataset: This dataset, containing Twitter users labeled as Control, Depressed, and PTSD, was used for training and initial evaluation. It provided a substantial amount of data for developing and comparing models.
  • Bell Let's Talk Dataset: This dataset was used for evaluating the generalization ability of the trained models. It consists of users labeled as either Depressed or Control and is noted as being imbalanced and not following the same age/gender distribution as the training set.

Key Insights and Ideas

  • Optimized Embeddings are Crucial: The most significant finding is that their custom-optimized word embeddings, trained using a multi-task approach on health-related data, consistently outperform standard pre-trained or random embeddings across various deep learning architectures. This highlights the importance of domain-specific embeddings for mental health detection.
  • CNNs Outperform RNNs for this Task: Surprisingly, the CNN-based models, particularly CNNWithMax, generally achieved better performance than the RNN-based models (BiLSTM with attention) for depression detection in this study. This suggests that for capturing salient features in short social media posts, the local feature extraction of CNNs might be more effective than the sequential modeling of RNNs, especially with limited data.
  • User-Level Detection is Feasible: The research successfully demonstrates that user-level depression detection (classifying an entire user based on their posts) is achievable with deep learning, even with relatively small and imbalanced datasets.
  • Attention Mechanisms Enhance RNNs: While CNNs performed better overall, the inclusion of a context-aware attention mechanism in the BiLSTM model did improve its performance compared to a basic BiLSTM.

Key Results

The paper reports strong performance, particularly with their optimized embeddings:

  • On the CLPsych 2015 dataset, the CNNWithMax model with optimized embeddings achieved an AUC of 0.951, with accuracy, F1, precision, and recall all above 86%. This significantly outperformed baseline models and other deep learning configurations.
  • On the Bell Let's Talk dataset (for generalization), the CNNWithMax model with optimized embeddings achieved an AUC of 0.920, with accuracy of 81.818%.
  • The study found that CNN-based models generally reported better performance than RNN-based models. For instance, the best RNN model reported an AUC of 0.832.

Limitations and Future Work

  • Dataset Size and Imbalance: The authors acknowledge that the CLPsych 2015 dataset, while useful, is relatively small, especially for training deep learning models. The Bell Let's Talk dataset is also imbalanced. This limitation might affect the robustness and generalization of the models.
  • Focus on Depression: The current work primarily focuses on depression. Future work could extend to detecting other mental disorders like PTSD, as mentioned in the paper.
  • More RNN Exploration: The authors suggest evaluating against more RNN-based models, particularly those incorporating more advanced attention mechanisms, to see if they can match or surpass CNN performance.
  • Ethical Considerations: While not a direct limitation of the model, the paper touches upon the ethical implications of using social media data, including privacy concerns and the need for anonymization, which are crucial for future research in this area.

Summary updated

BibTeX
@inproceedings{orabi2018deep,
  author = {Ahmed Husseini Orabi and Prasadith Buddhitha and Mahmoud Husseini Orabi and Diana Inkpen},
  title = {{Deep Learning for Depression Detection of Twitter Users}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
}
Figure