TL;DR
This paper introduces a novel methodology for generating a corpus of psychotherapy dialogues, specifically focusing on follow-up conversations in the context of Cognitive Behavioral Therapy (CBT). By leveraging personal narratives and an automated stimulus generation process, they create a valuable dataset for training more empathetic and effective conversational agents in the mental health domain.
What Problem Does This Paper Address?
Developing effective conversational agents for mental health support is a significant challenge. Current approaches often rely on rule-based systems or simple dialogue flows, leading to shallow interactions and limited user engagement. This is largely due to the complexity of mental health conversations, which involve personal feelings, user-specific situations, and nuanced emotional states. Existing dialogue corpora, often collected for tasks like information retrieval or slot-filling, are not well-suited for the deeply personal and longitudinal nature of mental health support. Specifically, there's a lack of data that captures the follow-up nature of therapeutic interventions, where agents need to recall past interactions and monitor patient progress over time.
Methods
The core of this paper is a multi-stage methodology for generating psychotherapy dialogues:
- Collection of Personal Narratives: Users receiving CBT were asked to write notes about daily events that triggered emotional arousal, using a mobile application that guided them through the ABC (Antecedent, Belief, Consequences) technique. This technique helps users understand how their thoughts influence their emotions.
- Stimulus Generation: An automatic algorithm was developed to create textual stimuli for dialogue writers. This algorithm extracts a key sentence from the personal narratives, incorporates the associated emotion (either user-provided or automatically detected via sentiment analysis), and uses predefined templates to generate a "common-ground statement" followed by a follow-up question. This mimics how psychotherapists might initiate a review of a patient's progress.
- Dialogue Generation: The generated stimuli were presented to two groups of dialogue writers: psychotherapists experienced in CBT and non-expert writers. Both groups were given detailed guidelines and examples to write follow-up dialogues, impersonating both the Personal Healthcare Agent (PHA) and the patient. The goal was to create dialogues that elaborated on the event introduced in the stimulus and included a closure turn.
Datasets
The primary dataset introduced in this paper is a corpus of follow-up psychotherapy dialogues. This corpus was generated using the described methodology and consists of 800 dialogues. The authors also mention using a lexicon-based sentiment analyzer from The OpeNER project for emotion detection and a summarizer from the sumy library for stimulus generation.
Key Insights and Ideas
- The importance of longitudinal data: The paper highlights that mental health support requires dialogues that span over time, mirroring real therapeutic interventions, which is a significant gap in current dialogue corpora.
- Automated stimulus generation bridges the gap: The proposed algorithm for generating stimuli from personal narratives is a key innovation, enabling the creation of contextually relevant prompts for dialogue writers.
- Domain expertise matters, but guidelines can compensate: While psychotherapists showed more nuanced validation of stimuli, non-expert writers, with appropriate guidelines, could still produce a course-grained validation, suggesting a scalable approach to data collection.
- Differences in dialogue writing: Experts tended to write shorter conversations but used a richer vocabulary, while non-experts produced more verbose dialogues with less lexical diversity. This suggests that domain expertise influences conversational style.
- Dialogue Acts reveal structural differences: Professionals used a more structured set of dialogue acts, focusing on core therapeutic functions, whereas non-experts incorporated more varied, sometimes less relevant, dialogue acts.
Key Results
- Corpus Size: 800 follow-up dialogues were collected.
- Stimuli Validation: Psychotherapists had a higher validation rate for stimuli (44.5% invalid) compared to non-experts (34.2% invalid), indicating domain expertise leads to more precise assessment.
- Dialogue Length: Experts wrote shorter dialogues (average 3.7 turns) than non-experts (average 4.2 turns).
- Vocabulary: Experts used a wider range of unique tokens (4251) compared to non-experts (3146).
- Response-Selection Model Evaluation: A baseline TF-IDF response-selection model trained on this corpus achieved promising results. Human evaluation showed that 91% of system turns were considered appropriate and coherent, leading to 72% of dialogues being rated as acceptable quality.
Limitations and Future Work
- Anonymization: The authors note that they are seeking further funding to anonymize the corpus for public release, respecting patient privacy.
- Scalability of Expert Writers: While psychotherapists are valuable, their availability is limited. The paper suggests that training a larger number of non-expert writers with appropriate guidelines could be a more cost-effective and scalable approach for data collection.
- Further Model Development: The current work focuses on generating the corpus and evaluating a baseline response-selection model. Future work could involve training more sophisticated dialogue models, including end-to-end systems, using this dataset.
- Broader Applicability: The methodology could be extended to other domains requiring personal, longitudinal conversations.
Summary updated