ExTRA: Explainable Therapy-Related Annotations

Mat Rawsthorne, Tahseen Jilani, Jacob Andrews, Yunfei Long, Jérémie Clos, Samuel Malins, Daniel HuntNL4XAI@INGL 2020 · OpenReview

TL;DR

This paper introduces ExTRA, a novel Explainable AI (XAI) tool that uses Natural Language Processing (NLP) to automatically label psychotherapy session transcripts based on patient activation levels. The goal is to help therapists assess session effectiveness and build trust in AI-driven insights through co-design and transparency.

What Problem Does This Paper Address?

Quality assurance in psychotherapy is a manual and time-consuming process, making routine assessment of session quality rare. This limits therapists' ability to objectively review their effectiveness and tailor care. Existing methods for analyzing therapy sessions are labor-intensive. The paper addresses the need for an automated tool that can provide objective, actionable feedback to therapists, helping them understand and improve their practice by identifying key interaction patterns predictive of client recovery. Furthermore, it tackles the challenge of building trust in AI tools within a sensitive domain like mental healthcare, where understanding why a model makes a prediction is crucial for adoption.

Methods

The core of the ExTRA tool utilizes a supervised machine learning approach, specifically the Chi-square Automatic Interaction Detection (CHAID) algorithm. CHAID is a decision tree-based classification model well-suited for handling both categorical and numerical data and does not require strict statistical assumptions like normality. To address class imbalance in the data (where some categories are much rarer than others), the researchers employed weighted sampling, giving more importance to underrepresented categories. For model validation, they used K-fold cross-validation. The project also emphasizes a co-design approach, involving a diverse team including a Service User Researcher, Involvement Volunteers, Clinical Psychologists, Corpus Linguists, and importantly, a Patient & Practitioner Reference Group (PPRG). This collaborative process aims to ensure the tool's usability and trustworthiness.

Datasets

The study used a corpus of 120 health anxiety therapy sessions. These transcripts were professionally annotated and then further processed using a coding scheme called the Clinical Interaction Coding Scheme (CICS). The CICS categorizes therapist-client interactions into seven types and rates them on a scale of -2 to +2 based on the degree of patient activation. This dataset is used for both training and evaluating the CHAID classifier.

Key Insights and Ideas

  • The power of co-design: Involving patients, carers, and clinicians directly in the development process is highlighted as crucial for building trust and ensuring the tool's practical utility. This "Responsible Research & Innovation" (RRI) approach aims to make the AI acceptable and useful.
  • Balancing accuracy and explainability: The project explicitly aims to understand the trade-offs between how accurate a model is and how well its reasoning can be explained. This is a fundamental challenge in XAI, especially in high-stakes domains.
  • Language as a window into therapy: The research demonstrates that specific language patterns within therapy sessions can be predictive of client outcomes, suggesting that NLP can unlock valuable insights into therapeutic effectiveness.
  • The importance of "why": The paper underscores that for AI tools to be adopted in clinical settings, therapists need to understand why the tool is making a particular suggestion or prediction. This goes beyond just providing an answer.

Key Results

The initial CHAID model achieved an overall accuracy of 74% in classifying patient activation levels into negative, neutral, and positive categories. The model performed best on the "Neutral" category. While the "Negative" category had a relatively low correct classification rate (60.4%), this was attributed to its small sample size (2.6% of the total data). The paper also notes that further improvements could be made through more advanced NLP techniques like SMOTE and deep learning models.

Limitations and Future Work

The authors acknowledge that the current model's performance could be enhanced with more sophisticated NLP techniques and deeper machine learning modeling. Future work includes refining the classification tool, evaluating different XAI approaches for transparency, and exploring whether direct stakeholder involvement improves usability and trust. They also plan to investigate how to augment the model using Argumentation Theory and explore related NLP developments for mental health applications. The project aims to test whether Natural Language Generation (NLG) can be used to provide explanations in plain English, tailored to different user perspectives.

Summary updated

Abstract

In this paper we report progress on a novel explainable artificial intelligence (XAI) initiative applying Natural Language Processing (NLP) with elements of codesign to develop a text classifier for application in psychotherapy training. The task is to produce a tool that will facilitate therapists to review their sessions by automatically labelling transcript text with levels of interaction for patient activation in known psychological processes, using XAI to increase their trust in the model’s suggestions and client trajectory predictions. After pre-processing of the language features extracted from professionally annotated therapy session transcripts, we apply a supervised machine learning approach (CHAID) to classify interaction labels (negative, neutral, positive). Weighted samples are used to overcome class imbalanced data. The results show this initial model can make useful distinctions among the three labels of patient activation with 74% accuracy and provide insight into its reasoning. This ongoing project will additionally evaluate which XAI approaches can be used to increase the transparency of the tool to end users, exploring whether direct involvement of stakeholders improves usability of the XAI interface and therefore trust in the solution.
BibTeX
@inproceedings{rawsthorne2020extra,
  author = {Mat Rawsthorne and Tahseen Jilani and Jacob Andrews and Yunfei Long and Jérémie Clos and Samuel Malins and Daniel Hunt},
  title = {{ExTRA: Explainable Therapy-Related Annotations}},
  booktitle = {NL4XAI@INGL},
  year = {2020},
  url = {https://openreview.net/forum?id=FBNnxjaZO2},
}
Figure