TL;DR
This paper introduces DeepWell-Adol, a novel, large-scale Chinese dialogue corpus specifically designed to promote positive mental health and well-being in adolescents. By combining expert-written dialogues with an automated augmentation framework, DeepWell-Adol offers a high-quality, scalable resource that addresses a critical gap in NLP research for adolescent mental health, moving beyond traditional clinical approaches.
What Problem Does This Paper Address?
Adolescence is a crucial developmental stage marked by significant emotional, cognitive, and social changes. While many NLP efforts in mental health focus on clinical therapy or treating disorders, there's a pressing need for resources that promote positive mental health, personal growth, and resilience in non-clinical adolescent populations. Existing mental health corpora often lack domain-specific relevance for this age group and may not incorporate strengths-based approaches from positive psychology and coaching. Furthermore, current evaluation metrics are insufficient for capturing the nuances of positive psychological outcomes in dialogue.
Methods
The authors developed a framework called DeepWell-Adol, which has two main components:
- Expert-Written Seed Corpus: 51 experts in adolescent mental health and positive psychology contributed 1,139 multi-turn dialogue samples. These dialogues were grounded in real-world scenarios and a Positive Mental Health and Wellbeing Promotion Framework (PMW), which includes dialogue construction guidelines (PMW-C) and outcome evaluation metrics (PMW-E).
- Automated Augmentation Framework (DeepSynergy): This framework uses a dual-phase approach to generate a large corpus of high-fidelity dialogues.
- Positive Psychological Promotion Plans Generation: A supervisor LLM designs scenario-specific intervention plans based on the PMW-C framework and expert dialogues.
- Positive Psychological Promotion Dialogue Generation: A coach LLM operationalizes these plans by generating multi-turn dialogues, incorporating evidence-based psychological tools.
This approach balances data quality (from experts) with quantity and diversity (from automated generation).
Datasets
- DeepWell-Adol Corpus: The primary contribution is this newly created corpus, comprising 1,795 high-quality multi-turn dialogues covering five key themes: emotion regulation, academic & career development, social & interpersonal relationships, lifestyle & environmental adaptation, and personal growth & self-identity. It includes both expert-written and computer-generated dialogues.
- Comparison Datasets: For evaluation, DeepWell-Adol was compared against two publicly available datasets: SMILECHAT (a multi-turn mental health support dialogue corpus) and CPsyCounD (a multi-turn psychological counseling corpus).
Key Insights and Ideas
- Bridging the Gap: The paper successfully demonstrates that it's possible to create a large-scale, high-quality dialogue corpus for adolescent positive mental health promotion by combining expert knowledge with automated generation. This addresses a significant gap in current NLP resources.
- Scalability with Quality: The DeepSynergy framework is a key innovation, showing how to scale up data creation without sacrificing the professional quality and domain relevance crucial for sensitive areas like mental health.
- Holistic Evaluation: The development of the PMW-E framework highlights the inadequacy of traditional NLP metrics for mental health tasks and proposes a more comprehensive approach that assesses positive psychological outcomes.
- Expert-AI Synergy: The study confirms that combining expert-written data with computer-generated data leads to superior performance in fine-tuned LLMs, suggesting a powerful synergy for future dataset development.
Key Results
- Corpus Performance: Evaluations showed that DeepWell-Adol performed comparably or better than SMILECHAT and CPsyCounD in general counseling competence and emotional support. Crucially, it demonstrated significantly stronger performance across key indicators of the PMW-E framework, highlighting its superiority in promoting positive mental well-being.
- Model Fine-tuning: Fine-tuning LLMs (Qwen2.5-7B, ChatGLM3-6B, Baichuan2-7B) on the DeepWell corpus resulted in substantial gains in ROUGE, BLEU-4, and BERTScore, indicating improved contextual relevance and semantic alignment.
- Ablation Studies: Removing either expert-written or computer-generated data led to performance drops, confirming the value of both components.
- Manual Evaluation: In a manual evaluation by experts, responses generated by their fine-tuned model (WellChat) were preferred in 54.5% of dialogue turns, compared to 27.9% for the baseline model and 17.5% for the human-written reference. This preference was statistically significant.
Limitations and Future Work
- Limited Cultural Diversity: The current corpus does not fully capture the psychological diversity across cultural and developmental contexts, which limits model generalizability.
- Static Knowledge: The generation process relies on static expert knowledge and predefined theme seeds, falling short of modeling dynamic, context-sensitive strategies.
- Future Directions: The authors suggest exploring more adaptive and temporally-aware generation approaches, strengthening filtering mechanisms for unsafe outputs, and extending the framework to other populations like the elderly. They also propose future work involving reinforcement learning from human feedback (RLHF).
Summary updated