Overview of the CLPsych 2025 Shared Task: Capturing Mental Health Dynamics from Social Media Timelines

Talia Tseriotou, Jenny Chim, Ayal Klein, Aya Shamir, Guy Dvir, Iqra Ali, Cian Kennedy, Guneet Singh Kohli, Anthony Hills, Ayah Zirikly, Dana Atzil-Slonim, Maria LiakataNAACL 2025

TL;DR

This paper introduces the CLPsych 2025 Shared Task, a novel challenge focused on understanding the dynamic nature of mental health by analyzing social media timelines. By combining evidence extraction, well-being prediction, and summarization, the task aims to build more nuanced and explainable models for mental health monitoring, moving beyond static assessments.

What Problem Does This Paper Address?

Mental health is a complex, dynamic process, not a static state. Traditional approaches often rely on periodic assessments, missing the subtle shifts and fluctuations that characterize an individual's well-being over time. Social media offers a rich, longitudinal source of data where people openly discuss their mental states. However, extracting meaningful insights from these vast, unstructured timelines presents a significant challenge. Existing NLP methods for mental health often focus on classification tasks or user-level analysis, failing to capture the intricate interplay of adaptive and maladaptive states that evolve within an individual's daily life. This shared task aims to bridge this gap by developing models that can not only detect mental health indicators but also explain how these states change and why, fostering more personalized and timely interventions.

Methods

The CLPsych 2025 Shared Task is structured around four key subtasks, all grounded in the MIND transtheoretical framework:

  • Task A.1: Evidence Extraction: This task involves identifying specific text spans within social media posts that indicate either adaptive or maladaptive self-states. This moves beyond simple sentiment analysis to pinpointing nuanced expressions of psychological functioning.
  • Task A.2: Well-Being Score Prediction: Participants are asked to assign a well-being score (1-10) to each post, reflecting the individual's social, occupational, and psychological functioning. This score is informed by the extracted evidence of adaptive and maladaptive states.
  • Task B: Post-level Summarization: This subtask requires generating a summary for each individual post, capturing the interplay between adaptive and maladaptive states and identifying the core aspects (Affect, Behavior, Cognition, Desire) driving these states.
  • Task C: Timeline-level Summarization: The ultimate goal is to summarize the temporal dynamics of self-states across an entire timeline, highlighting how these states evolve, fluctuate, and transition over time.

The task leverages a dataset of Reddit posts from mental health-related subreddits, annotated by clinical psychology experts using the MIND scheme. This scheme breaks down experiences into core building blocks (Affect, Behavior, Cognition, Desire - ABCD) that co-activate to form self-states.

Datasets

The dataset used for this shared task is derived from the Reddit-New dataset, originally introduced in CLPsych 2022. This dataset comprises user timelines extracted from mental health-related subreddits between 2015 and 2021. To manage the annotation load, timelines were selectively sampled for length, content relevance, user uniqueness, and "density diversity" (based on prior annotations of mood switches and escalations). The dataset contains annotated posts with information on adaptive/maladaptive self-states, ABCD elements, and well-being scores.

Key Insights and Ideas

  • The Asymmetry of Negativity: A striking finding is that current LLMs, especially smaller ones, are better at identifying and synthesizing negative (maladaptive) self-states compared to positive (adaptive) ones. This aligns with psychological theories suggesting negative experiences are often more salient and attention-grabbing.
  • Temporal Dynamics Matter: The task highlights the critical need for models that can capture temporal patterns in mental health. Simply analyzing individual posts in isolation is insufficient; understanding the evolution of self-states over time is key.
  • Explainability is Crucial: The emphasis on evidence extraction and summarization underscores the importance of not just predicting mental health states but also providing humanly understandable rationales. This is vital for clinical adoption and trust.
  • LLMs are Dominant, but Nuance is Challenging: While LLMs are the go-to tools for this task, achieving top performance often involves sophisticated prompting, retrieval augmentation, or fine-tuning. Identifying subtle adaptive states remains a significant challenge.

Key Results

  • Evidence Extraction: Top-performing systems achieved high recall, with some models demonstrating better performance in identifying maladaptive evidence spans than adaptive ones.
  • Well-being Score Prediction: The best systems achieved low Mean Squared Error (MSE), with performance varying across different well-being score ranges. Systems that integrated evidence extraction or jointly predicted evidence and scores tended to perform better.
  • Summarization: Systems generally achieved good consistency in their summaries, though the paper notes that models may struggle to synthesize positive aspects as effectively as negative ones.
  • LLM Trends: The shared task saw a significant increase in the diversity of LLM families used, a substantial increase in context length capabilities (up to 1M tokens), and a widespread adoption of LLMs across all tasks.

Limitations and Future Work

  • Generalizability: The current work is limited to English-language social media posts from self-selected online communities. Generalizing these findings to other contexts and languages remains an open research question.
  • Annotation Scope: The annotation process focused on identifying the most salient evidence spans. Future work could explore the nuanced selection of additional evidence spans and their connections.
  • Multimodality: Mental health manifestation often involves multimodal cues (e.g., tone of voice, facial expressions) that are not captured in text-only data. Incorporating multimodal information is a promising future direction.
  • Temporal Modeling: While progress has been made, further research is needed to develop more robust temporal models that can capture sparser and subtler changes over longer periods, especially for adaptive behaviors.
  • Clinical Utility: The authors emphasize that the extracted evidence and summaries cannot be used for diagnostic purposes directly. They should be reviewed by clinical experts or used to augment their capacity by efficiently presenting information. Future work could explore making causal links between evidence and well-being scores.

Summary updated

BibTeX
@inproceedings{tseriotou2025overview,
  author = {Talia Tseriotou and Jenny Chim and Ayal Klein and Aya Shamir and Guy Dvir and Iqra Ali and Cian Kennedy and Guneet Singh Kohli and Anthony Hills and Ayah Zirikly and Dana Atzil-Slonim and Maria Liakata},
  title = {{Overview of the CLPsych 2025 Shared Task: Capturing Mental Health Dynamics from Social Media Timelines}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure