TL;DR
This paper introduces a novel multimodal framework for detecting depression from clinical interviews that explicitly tackles interviewer bias. By employing adversarial learning and a new "Dialogue-based Contextual Positional Encoding," the model learns to focus on participant responses rather than superficial cues from interviewer questions, leading to more robust and fair depression detection.
What Problem Does This Paper Address?
Diagnosing depression often relies on clinical interviews, which involve a series of question-and-answer (QA) pairs. While multimodal approaches (using audio, text, and video) have improved depression detection accuracy, many models inadvertently learn to associate specific interviewer question patterns with depression. This "interviewer bias" can lead to inflated performance metrics on certain datasets but poor generalization to real-world, less structured interviews where questions are more generic. The core problem is building depression detection models that are genuinely participant-focused and not susceptible to these "shortcut learning" biases.
Methods
The authors propose a multimodal Dialogue-level Transformer architecture. Key innovations include:
- Dialogue-based Contextual Positional Encoding (D-COPE): This novel encoding method goes beyond standard positional embeddings by incorporating both the turn's position and the semantic context of the interviewer's question. This helps the model understand the dialogue's flow and the specific meaning of each turn.
- Adversarial Learning with Gradient Reversal Layer (GRL): To mitigate interviewer bias, an adversarial classifier is trained to predict "Interviewer Question Functions" (IQFs) – categories describing the purpose of an interviewer's question (e.g., "open-ended," "supportive"). The GRL ensures that while the main model learns to predict depression, it simultaneously learns representations that are invariant to these IQFs, thus reducing reliance on question patterns.
- Multimodal Feature Extraction: The model uses XLM-ROBERTa for text (interviewer questions and participant transcripts) and Wav2Vec2-XLSR-53 for audio (participant responses). These are fused using a gated mechanism.
- Dialogue Transformer: A lightweight transformer encoder processes the sequence of D-COPE enhanced representations to capture interdependencies between turns and generate contextualized embeddings for the entire interview.
Datasets
The study utilizes three real-world datasets and one synthetic dataset:
- DAIC-WOZ: An English-language dataset with 189 interviews, featuring audio, transcribed text, and visual data. Participants are scored using the PHQ-8.
- EATD: A Chinese dataset with 162 interviews, focusing on audio and text. Participants are labeled using the Self-Rating Depression Scale (SDS).
- Androids Corpus: An Italian dataset with 116 interviews, including audio and text. Participants are diagnosed using DSM-5 criteria.
- DAIC-Synthetic: Synthetically generated text data based on the DAIC-WOZ dataset, created using an LLM (GPT-40) to augment training data and improve generalization.
Key Insights and Ideas
- Interviewer questions can be a double-edged sword: While incorporating interviewer questions can boost performance, it also introduces a significant risk of bias. Models can learn to exploit question patterns rather than participant responses.
- Generic interviews are harder: Models that perform well on structured datasets like DAIC-WOZ struggle on less structured ones (EATD, Androids) where interviewer questions are more generic, highlighting the generalizability issue.
- Adversarial training is key for fairness: The adversarial component, by forcing the model to ignore question function labels, is crucial for developing representations that are truly participant-focused and less prone to bias.
- D-COPE is essential for dialogue context: The proposed D-COPE significantly improves the Dialogue Transformer's ability to understand sequential and contextual information within interviews.
Key Results
- The proposed model achieves competitive performance across all datasets, with particularly strong results on the DAIC-WOZ and DAIC-Synthetic datasets.
- Ablation studies show that removing D-COPE leads to sharp declines in performance, confirming its importance.
- The adversarial component (GRL) significantly reduces reliance on interviewer question functions, promoting fairness, even if it slightly impacts raw in-distribution accuracy.
- Multimodal approaches including interviewer prompts generally perform better on DAIC-WOZ, but their advantage diminishes on datasets with more generic prompts.
Limitations and Future Work
- Data Size: Available labeled clinical speech datasets are often small and privacy-restricted. The authors used oversampling and synthetic data, but larger, more diverse datasets are needed.
- Modality Scope: The study primarily uses text and audio. Integrating visual and physiological signals could further enrich multimodal models.
- Model Generalizability: The current framework is tailored for dyadic clinical interviews. Adapting it to other formats like monologues or self-reports presents a challenge.
- LLM Annotation Nuance: While LLMs are effective for labeling question functions, they may not fully capture the nuance of human interviewer behavior. Future work could involve human annotators to verify and expand upon LLM-generated labels.
Summary updated