Routine Outcome Monitoring in Psychotherapy Treatment Using Sentiment-Topic Modelling Approach

Noor Fazilla Abd Yusof, Chenghua Lin

TL;DR

This paper introduces a computational approach to Routine Outcome Monitoring (ROM) in psychotherapy by applying a Dynamic Joint-Sentiment-Topic (dJST) model to therapy transcripts. By automatically tracking how a patient’s sentiments and discussion topics evolve across sessions, the researchers demonstrate that machine learning can mirror—and potentially augment—the qualitative insights of professional psychotherapists.

What Problem Does This Paper Address?

In psychotherapy, tracking a patient's progress is vital for successful outcomes. Currently, clinicians rely on "Routine Outcome Monitoring" (ROM), which typically involves patients filling out standardized self-report questionnaires (like the OQ-45 or CORE) before and after sessions.

However, this traditional approach has several "pain points":

  1. Administrative Burden: It requires significant time and effort from both the patient and the clinician.
  2. Financial Cost: Administering and analyzing these tools can be expensive.
  3. Delayed Feedback: Often, these assessments act as a "post-mortem" analysis rather than providing real-time course correction during the treatment process.

The authors seek to bridge this gap by creating an inexpensive, automated tool that extracts progress indicators directly from the words spoken during therapy.

Methods

The core of the research is the Dynamic Joint-Sentiment-Topic (dJST) model. Unlike standard sentiment analysis (which just looks at "happy" vs. "sad") or standard topic modeling (which just looks at "what" is being discussed), dJST does both simultaneously over time.

  • Joint Modeling: The model assumes that topics and sentiments are intertwined. For example, a patient might discuss "family" (topic) with "negative" sentiment in early sessions but transition to "positive" sentiment later.
  • Dynamics (Temporal Tracking): The model treats each therapy session as an "epoch." It uses the word distributions from previous sessions to influence the "priors" (expectations) for the current session. This allows the model to track the evolution of a patient's mental state.
  • Lexicon Integration: The model is seeded with a sentiment lexicon to help it distinguish between positive and negative word usage effectively.

Datasets

The researchers utilized a historical but highly regarded dataset: the Carl Rogers psychotherapy transcripts.

  • Scope: 158 transcripts from 51 different clients of Carl Rogers (the founder of client-centered therapy).
  • Evaluation Subset: The authors focused on five specific cases (clients named Frank, Bryan, Marry, Vib, and Int) because these cases included detailed qualitative analysis published by professional psychotherapists, providing a "ground truth" for comparison.
  • Preprocessing: Transcripts were cleaned (lowercase, removal of non-alphanumeric characters and stop words), and only the client’s verbatim text was analyzed (the therapist's responses were excluded).

Key Insights and Ideas

  • NLP Mirrors Clinical Intuition: The most striking insight is how closely the model’s sentiment curves match the notes of a world-class therapist. For instance, in the case of "Herbert Bryan," the model correctly identified a significant "dip" into negative sentiment in session 5, which matched Rogers’ own notes about the patient’s discouragement and "neurotic satisfactions" during that specific interview.
  • Sentiment-Topic Correlation: The model doesn't just say a patient is "feeling better"; it identifies what they feel better about. It could distinguish between negative sentiments regarding "relationships" vs. positive sentiments regarding "educational goals."
  • The "Maturity" Link: For the case of "Miss Vib," the model’s upward trend in positive sentiment toward the final sessions correlated with an established "Index of Behavioral Maturity," suggesting that automated sentiment tracking could serve as a proxy for complex psychological maturity scales.

Key Results

  • High Accuracy: When comparing the model’s overall sentiment trend (positive vs. negative) against expert analysis, the model achieved 100% accuracy for four out of the five cases (Miss Vib, Miss Int, Bryan, and Marry).
  • Case Study Success: In the case of "Marry Jane Tilden," the model successfully tracked a dramatic decline in negative sentiment between sessions 7 and 9, aligning perfectly with the patient’s reported "fresh surge of insight" and "new choices."
  • Identifying Regressions: The model successfully flagged sessions where patients were "regressing" or feeling discouraged, which is the primary goal of ROM—allowing a therapist to adjust their style before the patient drops out of treatment.

Limitations and Future Work

  • The "Frank" Discrepancy: In one case (Frank, Session 5), the model identified negative sentiment while the expert saw positive progress. The authors suggest this might be due to the model misinterpreting specific words without enough clinical context.
  • Missing the Therapist’s Voice: This study only analyzed the client's words. Future work aims to integrate the counsellor’s verbatim transcripts, as the interaction between the two is likely a rich source of data for outcome prediction.
  • Predictive Limits: The authors noted that while the model is great at tracking the past and present, it is still difficult to predict the outcome of the next session, as human psychological progress is rarely linear.
  • Lexicon Specificity: The researchers suggest that incorporating more domain-specific mental health lexicons could further improve the model's nuance.

Summary updated

Abstract

Despite the importance of emphasizing the right psychotherapy treatment for an individual patient, assessing the outcome of the therapy session is equally crucial. Evidence showed that continuous monitoring of patient's progress could significantly improve the therapy outcomes to an expected change. The patient's progress can be tracked closely to help clinicians identify not progressing in the treatment by monitoring the outcome. This monitoring can help the clinician consider any necessary actions for the patient's treatment as early as possible, e.g., recommend different types of treatment or adjust the style of approach. Currently, the evaluation system is based on the clinical-rated and self-report questionnaires that measure patients' progress pre-and post-treatment. While outcome monitoring tends to improve therapy outcomes, there are many challenges in the current method, e.g., time and financial burden for administering questionnaires, scoring, and analyzing the results. Therefore, a computational method for measuring and monitoring patient progress throughout treatment is needed to enhance the likelihood of positive treatment outcomes. This paper focuses on developing a computational method using a Dynamic Joint-Sentiment-Topic model (dJST) to measure and monitor the patient treatment outcome by tracking patient’s current and recurrent views of topic and sentiment. We identified the sentiment and topic trend evolved throughout treatments for each therapy session on the author's level. Our results show that this computational method could potentially lead to an inexpensive clinical monitoring tool to evaluate patients' progress during psychotherapy.
BibTeX
@article{yusof2021routine,
  author = {Noor Fazilla Abd Yusof and Chenghua Lin},
  title = {{Routine Outcome Monitoring in Psychotherapy Treatment Using Sentiment-Topic Modelling Approach}},
  journal = {International Journal on Advanced Science Engineering and Information Technology},
  year = {2021},
  doi = {10.18517/ijaseit.11.6.13602},
}
Figure