TL;DR
This paper introduces a hierarchical recurrent neural network (RNN) with an attention mechanism for classifying social media posts related to mental health. The proposed model achieves improved classification performance compared to previous methods, particularly for less common mental health topics, and the attention mechanism offers valuable insights into which parts of the text are most indicative of a particular mental health issue.
What Problem Does This Paper Address?
Mental health is a significant public health challenge, and social media platforms offer a rich source of patient-generated data that can aid in understanding and addressing these issues. Automatically classifying this text to identify specific mental health problems is crucial for medical decision-making, public health policies, and improving healthcare. While Convolutional Neural Networks (CNNs) have shown promise, Recurrent Neural Networks (RNNs), which are better suited for sequential data like text, have been less explored for this task. Furthermore, understanding why a model makes a certain classification is vital for trust and deeper analysis, a capability that traditional methods often lack.
Methods
The authors employ a hierarchical Recurrent Neural Network (RNN) architecture inspired by Yang et al. (2016). This model processes text at two levels:
- Sentence Level: Bidirectional GRUs (Gated Recurrent Units) are used to encode words within each sentence, capturing sequential information.
- Document Level: Another set of bidirectional GRUs encodes the representations of sentences to form a document-level representation.
Crucially, the paper incorporates an attention mechanism at both the word and sentence levels. This mechanism allows the model to dynamically weigh the importance of different parts of the text when forming representations. Specifically, it learns to assign higher weights to words and sentences that are more relevant for the classification task. This attention mechanism is what allows the model to "focus" on key information, similar to how humans might read and interpret text.
Datasets
The study utilizes a dataset of posts from the social media platform Reddit. These posts are categorized into 11 mental health problems, forming a multiclass classification task. The dataset is described as having an imbalanced distribution across these topics, with "depression" being the most frequent and "autism" and "addiction" being the least frequent. The authors note that this dataset and its associated mental health topics were previously identified and used in other text classification studies.
Key Insights and Ideas
- RNNs are beneficial for sequential mental health text: The sequential nature of how individuals describe their experiences with mental health makes RNNs a natural fit, leading to improved classification over CNNs.
- Attention mechanisms provide interpretability: The attention weights can be visualized, offering insights into which sentences and words are most influential in the classification process. This is invaluable for understanding the model's decisions and for potential in-depth analysis.
- Focus on rare classes: The hierarchical RNN with attention showed significant improvements, especially for less represented mental health topics, suggesting its robustness in handling imbalanced datasets.
- Beginning and end of documents matter: The attention analysis revealed that the first and last sentences of a post tend to receive the most attention, indicating their importance in conveying key information.
Key Results
The proposed hierarchical RNN with attention (RNN-att) outperformed a baseline CNN architecture. Specifically:
- Overall Improvement: RNN-att achieved a 4 F-measure (FM) improvement over the CNN baseline.
- Performance on Rare Classes: The improvement was even more pronounced for rare classes, with an average of 8 FM improvement.
- Attention vs. Other Strategies: RNN-att also showed an average of 2.5 FM improvement over simpler aggregation strategies like maximum (RNN-max) and average (RNN-av) at the sentence level.
- Per-Class Gains: For specific classes like "schizophrenia," the performance improvement was substantial (up to 10 FM compared to CNN).
- Reduced Confusion: The RNN-att model demonstrated less confusion between similar mental health topics, for example, reducing the confusion between "schizophrenia" and "depression" by half compared to CNN.
Limitations and Future Work
- Computational Cost: RNNs generally require more computational power for training compared to CNNs.
- Future Directions:
- Applying the model to other types of health-related text, such as Electronic Health Records (EHRs), where event sequences are critical.
- Investigating word-level attention and comparing it with techniques like TF-IDF.
- Systematically comparing different attention mechanisms to find robust solutions that could potentially replace computationally expensive recursion.
- Exploring the use of attention weights to create more efficient, low-resource models by focusing on key sentences (e.g., the first two).
Summary updated