Abstract
In recent years, online communities have formed around suicide and self-harm prevention. While these communities offer support in moments of crisis, they can also normalize harmful behavior, discourage professional treatment, and instigate suicidal ideation. In this work, we focus on how interaction with others in such a community affects the mental state of users who are seeking support. We first build a dataset of conversation threads between users in a distressed state and community members offering support. We then show how to construct a classifier to predict whether distressed users are helped or harmed by the interactions in the thread, and we achieve a macro-F1 score of up to 0.69.
Introduction
Online mental health forums serve as a critical resource for individuals in crisis, offering immediate and anonymous peer support. However, these platforms are a double-edged sword. While they can provide a lifeline, they also risk the "Werther effect"—a phenomenon where exposure to self-harming behavior leads to contagion or normalization. This research addresses the challenge of quantitatively predicting a user's mental trajectory: can we determine if a distressed user will reach a stable state based on their interactions within a forum thread?
Methodology
The researchers approached this as a supervised classification problem. The core of the study involved analyzing conversation threads where a "target user" begins in a distressed state and receives replies from "participating users."
The pipeline consisted of:
- Thread Extraction: Identifying interactions starting with a distressed post.
- Automated Labeling: Using a high-performance classifier to label the final state of the target user as either "flagged" (distressed) or "green" (stable/helped).
- Symmetric Modeling: Instead of pooling all text, the researchers extracted features for the seeker and the helpers separately, hypothesizing that the specific interaction between these two groups is the primary driver of the outcome.
Data and Feature Engineering
The study utilized the 2016 CLPsych ReachOut.com dataset, comprising 64,079 posts from an Australian mental health forum. From this, 1,040 threads were extracted that met the criteria of having at least two posts from the target user and one reply.
Four main feature types were extracted:
- LIWC: Psycholinguistic categories like family, religion, or negative emotions.
- Sentiment & Subjectivity: Measuring emotional tone using TextBlob.
- Topic Modeling (LDA): Identifying latent themes such as "therapy/school" or "family."
- Textual Features: Standard TF-IDF weighted unigrams.
Key Insights
The most significant finding was that modeling the target user and the helpers separately significantly outperformed models that averaged their features. This confirms that the relationship between seeker and helper language is more predictive than the general tone of the thread.
Other key insights include:
- The Engagement Paradox: Counter-intuitively, the more a target user posted in a thread, the less likely they were to reach a "green" status. As shown in the data, threads with a high volume of posts from the seeker often fail to transition to a stable state, perhaps because participants struggle to connect effectively.
- Helpful Topics: Mentions of family, religion, or counseling by community members were associated with positive transitions.
- Linguistic Markers: High objectivity in a user's language correlated with decreased risk, while high subjectivity often signaled persistent distress.
Results
The researchers compared several classifiers, finding that Logistic Regression with ridge penalty performed best, achieving a macro-F1 score of 0.69. While LIWC features alone provided a strong baseline (0.64 F1), the inclusion of sentiment, LDA topics, and specific tokens was necessary to reach peak performance. XGBoost achieved the highest precision (75.7%), indicating high reliability when predicting that a user was successfully helped.
Conclusion and Future Work
While this study provides a robust framework for predicting peer support outcomes, it is limited by the short-term window of single threads and the inability to observe "lurking" or offline events. Future work could leverage larger datasets to employ deep learning architectures like Transformers to better model the temporal flow of these vital conversations.
Summary updated