TL;DR
This paper introduces a novel multi-task learning framework using Bi-LSTM to simultaneously detect shifts in user mood and predict suicidal risk from longitudinal social media posts. Their "all-in-one" approach achieved top performance on the CLPsych Shared Task 2022 for mood change detection, demonstrating the power of joint learning for related mental health tasks.
What Problem Does This Paper Address?
Mental health challenges, exacerbated by global events like COVID-19, have led to a surge in online self-reporting of mental well-being. This presents an opportunity to leverage social media data for early detection and monitoring of mental health issues. However, accurately identifying subtle changes in a user's mood over time and assessing their risk of self-harm from their textual data remains a significant challenge. Existing approaches often tackle these tasks independently or in a sequential manner, potentially missing synergistic benefits. This paper aims to bridge this gap by developing a unified model that jointly learns to detect moments of mood change and predict suicidal risk from a user's posting history.
Methods
The core of the proposed system is a Bidirectional Long Short-Term Memory (Bi-LSTM) network, a recurrent neural network architecture well-suited for capturing sequential dependencies in text. The authors employ a multi-task learning paradigm, where the Bi-LSTM is trained to perform two related tasks simultaneously:
- Task A: Moments of Change Detection: This task aims to identify specific points in a user's timeline where their mood shifts. The categories are "Switch" (sudden change), "Escalation" (gradual shift), and "None" (no change).
- Task B: Suicidal Risk Level Classification: Based on the detected mood changes, this task classifies the user's risk level as "Low," "Moderate," or "Severe."
To represent the user posts, the authors experiment with various feature embedding techniques:
- fastText: A word embedding model that captures semantic meaning by averaging word vectors within a post.
- Sentence-BERT (SBERT): A pre-trained model specifically designed for generating sentence embeddings, leveraging RoBERTa for enhanced semantic understanding.
- Task-specific Scores: Embeddings derived from pre-trained models for various text classification tasks (emoji, emotion, hate, irony, offensive, sentiment) are concatenated to provide additional discriminative features.
The model architecture involves preprocessing text, generating these multi-modal embeddings, feeding them into the Bi-LSTM for sequential representation, and finally using softmax classifiers for each task. They also investigate the impact of attention layers within the Bi-LSTM to allow the model to focus on more influential posts.
Datasets
The research utilizes the dataset provided for the CLPsych Shared Task 2022. This dataset consists of longitudinal user posts from social media platforms. The paper provides statistics on the training data, including the number of users, posts, and the distribution of labels for both Task A (Moments of Change) and Task B (Suicidal Risk Levels).
Key Insights and Ideas
- Synergy of Multi-task Learning: The paper strongly advocates for the benefits of multi-task learning in the mental health domain. By jointly learning related tasks (mood change and suicidal risk), the model can leverage shared representations and potentially improve performance on both compared to independent models.
- Effectiveness of Combined Embeddings: The combination of fastText, SBERT, and task-specific scores (referred to as
Ptask-emb) proved to be the most effective feature representation, outperforming simpler embedding strategies. This suggests that a richer representation capturing both general semantic meaning and specific linguistic cues is crucial. - Impact of Attention Mechanisms: The inclusion of attention layers in the Bi-LSTM model consistently improved performance across various metrics. This highlights the importance of allowing the model to dynamically weigh the importance of different posts within a user's timeline when making predictions.
- Handling Data Imbalance: The authors acknowledge the inherent class imbalance in mental health datasets (e.g., more "None" mood changes than "Escalation" or "Switch"). While they merged "No Risk" and "Low Risk" for Task B, the results indicate that class imbalance still influences performance, particularly for minority classes.
Key Results
- The proposed Multitask-attn-score model (using
Ptask-emband attention layers) achieved the top ranking for Task A (Moments of Change) on the CLPsych Shared Task 2022 test set, based on window-based and coverage-based metrics. - For Task B (Suicidal Risk Levels), the same model achieved the second-best result on the shared task, demonstrating strong performance across both tasks.
- The paper reports macro-averaged precision, recall, and F1 scores for both tasks, indicating that the model performs well across all classes, despite class imbalance. For Task A on the test set, the Multitask-attn-score model achieved a macro-averaged F1 score of 0.724. For Task B on the test set, it achieved a macro-averaged F1 score of 0.382.
Limitations and Future Work
- Class Imbalance: The authors acknowledge that the class imbalance in the dataset affects the model's sensitivity to minority classes. While they implemented strategies to mitigate this, further improvements are desired.
- Single-Task Comparison: The paper focuses on the multi-task approach. Future work could involve a more direct comparison with other state-of-the-art single-task learning models.
- User Information: The authors suggest exploring the explicit inclusion of user demographic information (e.g., age, gender) to potentially enhance the model's predictive capabilities for mental health and suicidal risk.
- Resampling Techniques: To further address class imbalance, the authors plan to investigate the use of resampling techniques in their pipeline.
Summary updated