User Dynamics and Thematic Exploration in r/Depression During the COVID-19 Pandemic: Insights From Overlapping r/SuicideWatch Users

Jianfeng Zhu, Ruoming Jin, Deric R. Kenne, NhatHai Phan, Wei‐Shinn KuJournal of Medical Internet Research 2024 · DOI

TL;DR

This study leverages NLP techniques, specifically BERT and BERTopic, to analyze user behavior and themes in the r/Depression and r/SuicideWatch subreddits during the COVID-19 pandemic. It reveals critical transition patterns, prevalent emotional expressions (sadness, disappointment), and key discussion topics, offering insights for improved mental health interventions.

What Problem Does This Paper Address?

The paper addresses the challenge of identifying and understanding individuals at risk of suicidal ideation (SI), particularly within the context of major depressive disorder. While depression is common and carries a significant suicide risk, not everyone with depression experiences suicidal thoughts. The COVID-19 pandemic exacerbated loneliness and mental health struggles, making early identification and intervention even more crucial. Existing research often focuses on individual subreddits or lacks a nuanced understanding of user transitions and overlapping behaviors between communities like r/Depression and r/SuicideWatch. This study aims to bridge this gap by examining the dynamics, emotional tones, and topics discussed by users who actively participate in both communities, seeking to inform better prevention and intervention strategies.

Methods

The researchers employed a multi-faceted approach combining natural language processing (NLP) and statistical methods. Key techniques include:

  • Data Collection: Archived posts from r/Depression and r/SuicideWatch subreddits spanning 2019-2022 were collected.
  • Trend and Survival Analysis: These methods were used to analyze user engagement patterns over time and to understand the duration of user transitions between the two subreddits.
  • BERT Family of Models: Pre-trained BERT models were utilized for sentiment and thematic analysis. Specifically, a fine-tuned DistilBERT model was developed and trained to classify suicidal ideation (SI) in posts. This involved a systematic process of data splitting, cross-validation, initial fine-tuning, manual validation, and iterative refinement.
  • BERTopic Modeling: This technique was applied to identify and cluster latent topics within the posts, allowing for a deeper understanding of the themes discussed by users, categorized by their SI labels.
  • GoEmotion Model: This model was used to extract emotion tone features from posts.
  • Correlation Analysis: Similarity scores were calculated between depression symptoms and emotional expressions to identify key relationships.

Datasets

  • Reddit Submissions: Archived posts from the r/Depression and r/SuicideWatch subreddits (2019-2022). This resulted in a dataset of over 150,000 posts from approximately 25,000 unique overlapping users.
  • Gold Standard Data Set: A dataset of 500 redditors, annotated by domain experts using the Columbia-Suicide Severity Rating Scale, with labels including supportive, suicide ideation, suicide behavior, suicide indicator, and suicide attempt. This dataset was expanded and used for fine-tuning the DistilBERT model.

Key Insights and Ideas

  • User Interdependence: The study highlights a significant overlap in users between r/Depression and r/SuicideWatch, suggesting these communities serve interconnected roles in users' mental health journeys.
  • The COVID-19 Turning Point: A notable shift in user engagement occurred around August 2020, when r/SuicideWatch surpassed r/Depression in post count, coinciding with the initial spread of COVID-19. This suggests the pandemic significantly impacted user needs and engagement patterns.
  • Emotional Landscape of Depression: Sadness emerged as the most prevalent emotion among overlapping users in r/Depression. Disappointment was also strongly correlated with key depression symptoms.
  • Key Depression Symptoms: Physical activity changes, negative self-view, and suicidal thoughts were identified as the most common depression symptoms, strongly correlating with disappointment.
  • Topic Clusters: BERTopic analysis revealed distinct topic clusters. For users identified with SI, "struggles with depression and motivation in school and work" was a prominent topic, alongside existential despair. For users without SI, "mental health and suicidal thoughts" was more prevalent, followed by family struggles.
  • Transition Dynamics: The median transition time from r/Depression to r/SuicideWatch shortened significantly in 2020 (26 days) compared to previous years, potentially due to pandemic-related stressors.

Key Results

  • User Overlap: Approximately 25,000 unique authors overlapped between the r/Depression and r/SuicideWatch communities.
  • Post Length Differences: Posts in r/SuicideWatch were shorter on average (373 words) compared to r/Depression (767 words).
  • SI Classification Accuracy: The fine-tuned DistilBERT model achieved an accuracy of 0.73 in classifying SI labels in r/SuicideWatch posts.
  • Prevalence of SI Content: Within the r/SuicideWatch subreddit, 62% of posts were categorized as indicative of SI.
  • Emotional Expression: Sadness (1.00), disappointment (0.78), and annoyance (0.57) were the most frequently expressed emotions in r/Depression posts.
  • Symptom Correlations: Physical activity changes (0.72), negative self-view (0.70), and suicidal thoughts (0.69) were the most correlated depression symptoms. Suicidal thoughts and negative self-view showed a strong correlation of 0.91.
  • Top Topics (SI Group): Existential despair (19%) and struggles with depression and motivation in school and work (12%) were the most discussed topics.

Limitations and Future Work

Limitations:

  • Data Source Specificity: The study relied solely on Reddit data, which may not fully represent the broader spectrum of individuals with mental health concerns and limits generalizability.
  • Model Accuracy: While the DistilBERT model showed good accuracy, further refinement might be needed.
  • Causality: The observational nature of the study prevents establishing direct causal relationships between variables.

Future Work:

  • Diversify Data Sources: Incorporate data from a wider range of online mental health platforms, forums, and social media.
  • Investigate Causal Relationships: Conduct surveys, interviews, or experiments to uncover factors influencing suicidal thoughts.
  • Targeted Interventions: Develop early-stage online mental health support initiatives and explore the integration of empathetic human chatbots and online community counseling.
  • Broader Context: Include a broader online community context in future research to generalize mental health issues related to SI.

Summary updated

Abstract

Our study underscores the effectiveness of using natural language processing techniques to explore language markers and patterns associated with mental health challenges in online communities like r/Depression and r/SuicideWatch. These insights offer novel perspectives distinct from previous research. In the future, there will be potential for further refinement and optimization of machine classifications using these techniques, which could lead to more effective intervention and prevention strategies.
BibTeX
@article{zhu2024user,
  author = {Jianfeng Zhu and Ruoming Jin and Deric R. Kenne and NhatHai Phan and Wei‐Shinn Ku},
  title = {{User Dynamics and Thematic Exploration in r/Depression During the COVID-19 Pandemic: Insights From Overlapping r/SuicideWatch Users}},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  doi = {10.2196/53968},
}
Figure