NLP meets psychotherapy: Using predicted client emotions and self-reported client emotions to measure emotional coherence

Neha Warikoo, Tobias Mayer, Dana Atzil-Slonim, Amir Eliassaf, Shira Haimovitz, Iryna Gurevych

TL;DR

This paper introduces an end-to-end NLP pipeline to measure emotional coherence—the alignment between a client's inner emotional state and their spoken words—across real-world psychotherapy sessions. Using an AlephBERT model trained on Hebrew therapy dialogues, the authors show that coherence in expressing positive emotions strongly correlates with client well-being ($r = 0.67$), providing empirical evidence for a long-standing theory in clinical psychology.

What Problem Does This Paper Address?

In psychotherapy, emotional coherence—the coordination among different emotional response channels over time—is widely considered essential for mental health. Clinicians theorize that when individuals put their internal emotional experiences accurately into words, they can better regulate affect and resolve psychological distress.

However, verifying this hypothesis has faced major bottlenecks:

  1. Artificial setups: Prior coherence research relied mostly on single-time-point, lab-induced emotional stimuli rather than genuine interpersonal interactions.
  2. Missing channels: Past studies primarily compared subjective reports against facial expressions or physiological signals (e.g., heart rate), neglecting the connection to natural language.
  3. Scalability: Manually annotating therapy sessions utterance-by-utterance is prohibitively expensive, restricting clinical research to tiny sample sizes.

This work bridges these gaps by applying transformer-based NLP to longitudinal psychotherapy dialogues to measure coherence between post-session self-reports and automated, utterance-level emotional speech.

Methods

The authors designed an end-to-end framework combining language modeling, self-report psychometrics, and correlation analysis:

  • Utterance-Level Emotion Classification: The authors fine-tuned AlephBERT (a Hebrew pre-trained BERT variant) to classify client utterances into emotional categories. Training utilized 10-fold cross-validation on expert-annotated sessions with hyperparameter tuning, L2 regularization, and partial class balancing to handle skewed emotion distributions.
  • Silver-Dataset Inference: The trained AlephBERT model automatically predicted utterance-level emotions across the remaining un-annotated sessions.
  • Measuring Emotional Coherence: Utterance-level counts for positive and negative emotions were normalized per session. These proportions were then correlated (using Pearson's $r$) with the client's post-session self-reported mood state.
  • Relating Coherence to Well-Being: At the individual client level, average session coherence was correlated against client-reported clinical well-being scores across the course of treatment.

Datasets

The study utilizes clinical data from an active university clinic monitoring routine psychotherapy:

  • BIU-872: A corpus of 872 transcribed Modern Hebrew therapy sessions involving two speakers (client and therapist).
  • BIU-872_Gold: 196 sessions annotated at the utterance level by clinical experts across four categories: Positive, Negative, Neutral, and Mixed.
  • BIU-872_Silver: The remaining 676 un-annotated sessions, automatically labeled by AlephBERT.
  • Profile of Mood States (POMS): A 12-item post-session self-report evaluating current mood, grouped into positive (calmness, contentment, vigor) and negative (anger, sadness, anxiety) sub-scales.
  • Outcome Rating Scale (ORS): A pre-session visual analog scale (0–10 across four items) completed by clients to track overall well-being and psychological functioning.

Key Insights and Ideas

  • Language Models Validate Clinical Theory: Automated predictions reliably matched human expert patterns, proving that NLP can detect genuine emotional signals at scale without relying entirely on manual coding.
  • The Asymmetry of Emotional Coherence: While coherence between internal state and verbal expression existed for both positive and negative emotions, only positive emotional coherence was significantly associated with client well-being.
  • Why Positive Coherence Matters: Negative emotions are naturally salient and readily communicated. In contrast, genuinely acknowledging and verbally expressing positive emotions may represent an adaptive psychological milestone that directly mirrors therapeutic progress and symptom relief.

Key Results

  • Model Performance: AlephBERT achieved a micro-F1 score of 0.66 on utterance-level emotion recognition in psychotherapy dialogue.
  • Internal vs. Expressed Coherence (Hypothesis 1 validated):
  • Gold set (human labels): Significant positive correlation between self-reported POMS and verbal utterances for positive ($r = 0.29, p = 7.8 \times 10^{-5}$) and negative emotions ($r = 0.24, p = 0.001$).
  • Silver set (AlephBERT predictions): Sustained significant correlation for positive ($r = 0.27, p = 4.3 \times 10^{-12}$) and negative emotions ($r = 0.21, p = 4.1 \times 10^{-8}$).
  • Coherence vs. Well-Being (Hypothesis 2 validated for positive affect):
  • Positive emotional coherence strongly and significantly correlated with ORS well-being: $r = 0.67$ ($p = 0.048$).
  • Negative emotional coherence showed no statistically significant association with well-being ($r = -0.37, p = 0.321$).

Limitations and Future Work

  • Coarse Emotional Buckets: The study aggregated nuanced emotional states into binary positive/negative composites, potentially masking discrete affective dynamics (e.g., anxiety vs. anger).
  • Moderate Classifier F1: The emotion classification model achieved an F1 of 0.66, highlighting the inherent ambiguity and difficulty of interpreting conversational psychotherapy text.
  • Language and Setting Specificity: The evaluation was conducted exclusively on Modern Hebrew transcripts from a single university clinic.
  • Future Directions: Real-time application of these models could flag sessions characterized by high emotional dissociation, helping clinicians tailor interventions to assist clients in naming and integrating their feelings.

Summary updated

Abstract

Emotions are experienced and expressed through various response systems. Coherence between emotional experience and emotional expression is considered highly important to clients' well being. To date, emotional coherence has been studied at a single time point using lab-based tasks with relatively small datasets. No study has examined emotional coherence between the subjective experience of emotions and utterance-level emotions over therapy sessions or whether this coherence is associated with clients' well being. Natural language Processing (NLP) approaches have been applied to identify emotions during psychotherapy dialogue, which can be implemented to study emotional processes on a larger scale and with specificity. However, these methods have yet to be used to study coherence between emotional experience and emotional expression over the course of therapy and whether it relates to clients' well-being. This work presents an end-to-end approach where we use emotion predictions from our transformer based emotion recognition model to study emotional coherence and its diagnostic potential in psychotherapy research. We first employ our transformer based approach on a Hebrew psychotherapy dataset to automatically label clients' emotions at the utterance level in psychotherapy dialogues. We subsequently investigate the emotional coherence between clients' self-reported emotional states and our model-based emotion predictions. We also examine the association between emotional coherence and clients' well being. The findings indicate a significant correlation between clients' self-reported emotions and positive and negative emotions expressed verbally during psychotherapy sessions. Coherence in positive emotions was also highly correlated with clients well-being. These results illustrate how NLP can be applied to identify important emotional processes in psychotherapy to improve diagnosis and treatment for clients who suffer from mental-health problems.
BibTeX
@inproceedings{warikoo2022nlp,
  author = {Neha Warikoo and Tobias Mayer and Dana Atzil-Slonim and Amir Eliassaf and Shira Haimovitz and Iryna Gurevych},
  title = {{NLP meets psychotherapy: Using predicted client emotions and self-reported client emotions to measure emotional coherence}},
  booktitle = {PAI4MH @ NeurIPS},
  year = {2022},
  url = {https://openreview.net/forum?id=Wv_b9WkpEuD},
}
Figure