TL;DR
This paper introduces C-GraphSAGE, a novel graph neural network model for detecting suicidality in social media posts. By leveraging a custom-built, context-aware suicide dictionary and a heterogeneous graph structure, the model significantly outperforms existing methods in accurately identifying suicide risk levels, paving the way for earlier intervention.
What Problem Does This Paper Address?
Detecting suicidality on social media is crucial for early intervention, but existing methods often struggle with the nuances of language. Many approaches rely on simple suicide dictionaries, which can misinterpret words out of context (e.g., "cut" in "cut my hair" vs. "cut my wrist"). This leads to inaccurate risk assessments. The paper addresses this by proposing a method that not only identifies suicide-related words but also understands their contextual meaning within social media posts, capturing the dynamic semantic relationships between posts and words.
Methods
The core of the proposed solution is the Contextual GraphSAGE (C-GraphSAGE) model. It builds a heterogeneous graph where nodes represent social media posts and words from a suicide dictionary. Edges connect posts to words they contain and words that co-occur within posts.
Key methodological contributions include:
- Contextualized Node Features: Posts are encoded using pre-trained BERT, while suicide words are represented by their risk level and Word2Vec embeddings. This captures both the post's overall meaning and the specific semantic properties of suicide-related terms.
- Heterogeneous Graph Construction: The graph models two types of relationships: post-word associations and word-word co-occurrences, allowing the model to learn complex dependencies.
- Contextualized Graph Convolutional Encoder: Instead of standard GraphSAGE aggregators, the paper employs a Convolutional Neural Network (CNN) for aggregation. This CNN is designed to capture local patterns and relationships among neighboring nodes, which is found to be more effective for this task than RNN-based aggregators.
- Custom Suicide Dictionary: The authors address the limitations of existing clinical suicide dictionaries by creating a new, word-level English suicide dictionary derived from social media data (specifically, Reddit's r/SuicideWatch and related subreddits). This dictionary is validated by clinical experts.
Datasets
- UMD Reddit Suicidality Dataset: Used to create the custom suicide dictionary. This dataset comprises 79,569 posts from 37,083 subreddits, labeled with suicidality severity (No risk, Low risk, Moderate risk, Severe risk) by crowdsourcing and domain experts.
- The Golden Standard Dataset: Used for training and evaluating the proposed model. This dataset consists of Reddit posts from suicide-related subreddits, annotated with five suicidality levels (Supportive, Indicator, Ideation, Behavior, Attempt) by mental health experts. The paper uses a stratified 60:20:20 split for train, validation, and test sets.
Key Insights and Ideas
- Context is King: The paper strongly emphasizes that simply matching words from a dictionary is insufficient. Understanding the context in which these words appear is critical for accurate suicidality detection.
- Social Media Language Matters: Clinical suicide dictionaries may not accurately reflect the language used on social media. Creating a dictionary from social media data itself leads to better performance.
- Graph Neural Networks Excel at Relational Data: The heterogeneous graph structure effectively captures the complex interplay between posts, words, and their semantic relationships, which is a strength of GNNs.
- CNN Aggregation for Local Patterns: Using a CNN-based aggregation function within the GNN encoder proved more effective than traditional aggregators (like Pool, LSTM, Mean) because it can better identify local structural patterns among related words.
Key Results
- Superior Performance: The proposed C-GraphSAGE model achieved the highest G-F1 score (0.84) among all evaluated models, outperforming both lexicon-based and deep learning baselines.
- Dictionary Effectiveness: The custom-built suicide dictionary, derived from social media data, significantly improved performance compared to a domain expert-created dictionary when used with rule-based and RF models (Table 3).
- Dictionary Integration Benefits: Deep learning models incorporating the suicide dictionary consistently outperformed their counterparts without one, demonstrating the value of contextual information.
- Qualitative Analysis: The model accurately identified lower risk levels in posts that contained high-level suicide words but were contextually less severe, showcasing its ability to avoid misinterpretations.
Limitations and Future Work
- Subjectivity and Bias: The authors acknowledge that social media analysis is inherently subjective and can be influenced by demographic, annotator, and media-specific biases.
- Generalizability: The model's performance might not generalize perfectly to other social media platforms due to potential differences in language use and data characteristics. The current study uses Reddit data.
- Data Overlap: There's a slight overlap in the time frames of the data used for dictionary creation and model training, though a Jaccard similarity analysis suggests the datasets are largely unrelated.
- Future Work: The paper suggests that an interpretable model like C-GraphSAGE can help in understanding and improving other targets with different statistical patterns and biases. Further research could explore its application to other platforms and languages.
Summary updated