TL;DR
CounselTraj is a novel framework that transforms short, "shallow" synthetic counseling dialogues into long-form, process-aware sessions by planning a structured trajectory of dialogue acts. By training on human role-play data, the system generates CounselTrajD, a dataset of over 10,000 high-quality Chinese counseling dialogues, and CounselTrajLM, a model that significantly outperforms existing open-source counseling LLMs in therapeutic understanding and session coherence.
What Problem Does This Paper Address?
Training LLMs for psychological counseling is a "data desert" problem. Authentic transcripts are nearly impossible to share due to strict privacy requirements. While researchers have turned to synthetic data, existing datasets suffer from three major "hallmarks of artificiality":
- Short Sessions: Most synthetic dialogues last only 5–15 turns, whereas a real session typically lasts an hour (30–40+ turns).
- Abrupt Disclosure: Synthetic clients often dump their entire life story in the first turn, missing the gradual "warming up" and relationship-building seen in real therapy.
- Rigid Patterns: Synthetic counselors often fall into a repetitive "empathy-then-question" loop, failing to use the diverse strategies (reflection, affirmation, suggestion) that human experts employ as a session evolves.
Methods
The authors propose a five-step pipeline to bridge the gap between brief synthetic "seeds" and professional-grade sessions:
- Dialogue-Act Annotation: They took high-quality human role-play data (from the KokoroChat dataset) and annotated every turn with specific "dialogue acts" (e.g., Reflection, OpenQuestion for counselors; SelfDisclosure, Resistance for clients).
- Brief Dialogue Construction: They used LLMs to "compress" these long human sessions into brief, information-dense summaries that mimic the style of existing synthetic datasets.
- Trajectory Planning Training: They trained a planner (based on Qwen3-8B) to predict the full sequence of dialogue acts (the "trajectory") based only on the brief summary.
- Trajectory Planning: For any new short synthetic dialogue, the planner generates a 40+ turn roadmap of what acts the counselor and client should perform next.
- Session-Level Expansion: Finally, a large model (GPT-4o or similar) uses the brief seed and the planned trajectory as "soft guidance" to write out the full, long-form dialogue.
Datasets
The study utilizes several key datasets for training and evaluation:
- Human-Collected: KokoroChat (Japanese role-play), AnnoMI (English), and PsyDial-D4 (Chinese).
- Synthetic Seeds: SmileChat, CPsyCounD, and PsyDTCorpus.
- New Contribution: CounselTrajD, a new benchmark containing 10,964 process-guided Chinese counseling dialogues, each averaging over 87 utterances.
Key Insights and Ideas
- The "Arc" of a Session: By analyzing human data, the authors found a distinct "increase-then-decrease" pattern in client utterance length. Real clients talk more as they open up in the middle of a session and less as the counselor moves toward summaries and coping strategies. Synthetic datasets usually show a flat or strictly decreasing trend.
- The Value of Resistance: Human clients often show "Resistance" (hesitation or disagreement), which is crucial for realistic training. Synthetic clients are often "too cooperative," making the counseling process look unrealistically smooth.
- Acts over Text: Planning the function of a sentence (the dialogue act) before the content prevents the model from drifting off-topic or becoming repetitive during long-form generation.
Key Results
The authors evaluated their fine-tuned model, CounselTrajLM, against nine other Chinese counseling models.
- Session Quality: In "LLM-as-a-judge" evaluations, CounselTraj-expanded dialogues scored significantly higher (avg. 4.24/5) than direct expansion (2.43/5) or the original brief seeds (2.36/5).
- Human Preference: Professional psychology students preferred CounselTraj-expanded dialogues over the original synthetic versions in over 70% of cases.
- Downstream Performance: CounselTrajLM achieved an overall score of 4.70 on the PsychEval benchmark, substantially exceeding the previous state-of-the-art, SoulChat2.0 (3.82). It showed particular strength in "Therapeutic Alliance" and "Process Coherence."
Limitations and Future Work
- Role-Play vs. Reality: The model is trained on role-play data between humans, which, while high-quality, may still differ from actual clinical encounters with real patients.
- Language Specificity: The current implementation and the resulting CounselTrajD dataset are focused on Chinese. The authors note that cross-lingual validation is a key step for future work.
- Clinical Validation: While the model performs well on linguistic and therapeutic benchmarks, it has not been clinically validated for use in real-world medical or crisis intervention settings.
Summary updated