Abstract
The Working Alliance Transformer (WAT) is a novel framework designed to classify psychiatric conditions by analyzing the "working alliance"—the collaborative relationship between patient and therapist—directly from dialogue transcripts. By projecting dialogue turns into the embedding space of clinical inventories, the model achieves improved diagnostic accuracy and provides more interpretable insights into therapeutic dynamics compared to standard text-based models.
Introduction
In psychotherapy, the "working alliance"—defined as the bond and agreement on goals and tasks between patient and therapist—is a primary predictor of successful outcomes. Traditionally, this alliance is measured via post-session self-report surveys, which are subjective and lack temporal granularity. From a machine learning perspective, identifying a patient's condition (e.g., anxiety vs. schizophrenia) from lengthy dialogue is a "needle in a haystack" problem. Standard NLP models often fail to capture the subtle clinical dynamics of the relationship. This research bridges the gap between raw natural language processing and established clinical psychology frameworks by moving from session-level summaries to turn-by-turn psychological state tracking.
Methodology
The authors propose a pipeline that transforms raw dialogue into clinically meaningful features, as illustrated in Figure 1.
The core innovation is the Psychological State Encoder. The model utilizes the Working Alliance Inventory (WAI)—a 36-item clinical tool—and encodes each item into a vector space. For every turn in a therapy session, an embedding is generated (using SentenceBERT or Doc2Vec). The model then calculates the cosine similarity between that turn and each of the 36 WAI items, resulting in a 36-dimensional "Working Alliance Score" for every interaction.
These scores are concatenated with the original text embeddings to create a "Working Alliance Embedding." These sequences are then processed by one of three backbone architectures:
- Transformer (WAT): Uses attention mechanisms to capture long-range dependencies.
- LSTM (WA-LSTM): A recurrent model suited for temporal sequence data.
- RNN (WA-RNN): A baseline recurrent model.
Datasets
The study utilized the Alexander Street Counseling and Psychotherapy Transcripts, comprising over 950 therapy sessions. The dataset includes transcripts of real-world sessions, 40,000 pages of client narratives, and 25,000 pages of reference works. Sessions are labeled with one of four psychiatric conditions: Anxiety, Depression, Schizophrenia, or Suicidal. Due to significant class imbalance (e.g., 495 anxiety sessions vs. 12 suicidal sessions), specific sampling techniques were employed during training to mitigate majority-class bias.
Key Results
The performance of the various models across the four conditions is summarized in Figure 3.
The results indicate that the WA-LSTM using only patient turns and the combined "Working Alliance Embedding" achieved the highest accuracy of 46.0% (using Doc2Vec). A significant finding is that adding the 36-dimensional working alliance score improved performance over raw text embeddings in nearly all cases. For instance, using Doc2Vec, the WA-LSTM improved from a 44.3% baseline to 46.0% with the alliance features.
Furthermore, the results suggest that patient turns are more informative than therapist turns, likely because patient dialogue contains more narrative diagnostic signals. While LSTMs achieved higher peak accuracy in some configurations, the Transformer (WAT) demonstrated greater stability during training and more consistent convergence, particularly when handling the long sequences typical of therapy sessions.
Discussion and Future Work
The integration of domain knowledge as a prior—specifically the WAI features—acts as a clinical inductive bias that "nudges" the model toward relevant features. This approach allows for the identification of granular dynamics, such as "ruptures" in trust or therapeutic "breakthroughs," in real-time.
However, limitations remain, particularly regarding data imbalance for high-risk cases like suicidal ideation. Future work aims to deploy these methods as real-time "companion tools" for therapists-in-training and to integrate reinforcement learning to recommend treatment strategies based on the inferred alliance state.
Summary updated