Prompt-Matched Entrainment as an Auditable Depression Signal

Clinical interview models for depression have a credibility problem. On the most widely used benchmark, the DAIC-WOZ interviews conducted by a virtual interviewer called "Ellie," a text model that reads only the interviewer's questions reaches an F1 of 0.88. The reason is that the interview script branches: follow-up questions are triggered by what the participant said, so the questions asked already encode the diagnosis (DAIC-WOZ: On the Validity of Using the Therapist's prompts). Those authors flag the obvious next question and leave it open: does the same leak exist in the audio? One acoustic signal is especially exposed to it. Prosodic entrainment is the tendency of speakers to drift toward each other's pitch, loudness and speaking rate. It is computed over pairs of interviewer and participant turns, so any entrainment score quietly inherits which interviewer turns occurred.

We propose to measure participant-to-interviewer entrainment in a way that holds the interviewer side fixed, then feed it to a model that is transparent by construction. The model is additive over named components such as "pitch proximity, participant toward interviewer." It admits nonlinear terms only when they beat a linear baseline. The aim is a depression signal whose explanation is exact rather than post hoc, and whose benchmark performance cannot come from the interview script. A colleague's two-sentence version: compare how a participant's voice tracks the interviewer on the questions everyone was asked, against a shuffled control from the same participant, so that neither the script nor the person's habitual voice can masquerade as entrainment. Then test whether that controlled, human-readable signal adds anything to depression detection beyond the participant's own prosody.

Core Claim

The virtual interviewer is an unusual gift for entrainment research. If Ellie's utterances come from a fixed repertoire, which we verify rather than assume by measuring acoustic variance across repeated instances of each prompt, then one side of the dyad is acoustically constant. Any adaptation must come from the participant. This makes directional entrainment identifiable in a way it is not with human therapists. That matters because direction is where the clinical signal lives: in suicide-risk interviews only patient-to-therapist entrainment tracked the therapeutic bond, while the reverse direction did not (Towards an Unsupervised Entrainment Distance).

Our claim has two parts. First, naive entrainment features on DAIC-WOZ absorb the script shortcut and will look predictive for the wrong reason. Second, prompt-matched entrainment, restricted to the core prompts shared by all participants and contrasted with a same-speaker sham pairing, either carries depression information beyond patient-only prosody or it does not. Both outcomes are publishable and clinically informative.

Research Program

Measure. For participant $i$, shared core prompt $q$, and prosodic dimension $d$ (mean and range of pitch, intensity, speaking rate, and the voice-quality measures jitter, shimmer and noise-to-harmonics ratio, extracted with Praat as in Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions), let $p_{iqd}$ be the participant's standardized feature on their response turn. Let $a_{qd}$ be Ellie's feature on the prompt. Features are z-scored within speaker to remove habitual voice, the "dyadosyncrasy" confound (``Dyadosyncrasy'', Idiosyncrasy and Demographic Factors in Turn-Taking). The prompt-matched proximity score is

$$e_{iqd} = -|p_{iqd} - a_{qd}| + \frac{1}{|Q|-1}\sum_{q' \neq q} |p_{iqd} - a_{q'd}|,$$

where $Q$ is the set of shared core prompts. The score is positive when the participant's response sits closer to the prompt they actually heard than to other prompts they also heard. This borrows the sham-conversation logic of A Discriminative Acoustic-Prosodic Approach for Measuring Local Entrainment, which contrasts real turn pairs with block-shuffled ones. We move the shuffle inside a participant and inside the fixed prompt set, so neither speaker identity nor prompt identity can differ between real and control pairs.

Analogous synchrony terms (turn-to-turn co-variation) and convergence terms (change over interview time) follow the three-level decomposition of the multi-level framework. The convergence terms address the non-stationarity concern raised by A Study on Prosodic Entrainment in Relation to Therapist Empathy in Counseling Conversation. Each participant ends up with roughly two dozen named entrainment features plus the matching patient-only prosodic features.

Model. We fit a reluctant additive model of the kind A reluctant additive model framework for interpretable nonlinear individualized treatment rules uses for treatment rules. The procedure starts from a sparse linear model, fits penalized splines only to its residuals, and admits a spline only if a complexity-penalized criterion favors it. We use this method because the literature says entrainment is unlikely to be monotone. Intensity matching accompanies both rapport and shouting matches (Quantification of prosodic entrainment in affective spontaneous spoken interactions of married couples), and stable versus fluctuating pitch entrainment pull observer and client ratings in opposite directions (Tao & Lee). The reluctance rule keeps those nonlinearities honest on a dataset of under 200 interviews. The model's explanation is its own additive decomposition, so there is no gap between what the model does and what the clinician is shown.

Audit. Every model is trained under two regimes. The "full" regime uses all turn pairs, including script-dependent follow-ups. The "core-only" regime uses only shared prompts. The drop between the two regimes is our acoustic analogue of the text shortcut that Burdisso et al. measured.

Baselines

The naive reference point is a majority-class predictor together with a label-permutation null on the same folds. With roughly 189 interviews, any F1 must clear the permutation distribution before it means anything.

The script-shortcut reference is a model that sees only which prompts occurred and their timing, with no participant audio. This acoustic-side analogue of E-GCN measures how much signal is available without any participant behavior. For the text version, the file reports F1 0.88 for interviewer-prompts-only and 0.90 for the prompt-plus-participant ensemble on DAIC-WOZ (DAIC-WOZ: On the Validity of Using the Therapist's prompts). Participant-text-only Longformer reaches a depression-class F1 of 0.64, and affect-lexicon hierarchical attention reaches 70.3 (Affective Conditioning on Hierarchical Attention Networks). We treat 0.64 as the shortcut-free text bar and 0.88–0.90 as the inflated ceiling.

The strongest competitor on the actual question is a patient-only prosody model built from the same Praat features. It is fit with the same reluctant additive procedure, the same core-only turns and the same nested cross-validation budget. It asks whether entrainment adds anything once the participant's own voice is accounted for. The literature's patient-only depression markers, reduced loudness and lower pitch found by attention back-tracing in An interpretable speech foundation model for depression detection, make this a serious opponent. The file contains no reported result on Lacuna for audio-only DAIC-WOZ binary depression detection. We will therefore establish this number ourselves rather than quote one.

The strongest high-capacity competitor is a black-box audio model: pooled HuBERT embeddings over participant turns with a small classifier, explained post hoc with SHAP. On E-DAIC regression, the reference figure is MLlm-DR's concordance correlation coefficient of 0.72 (MLlm-DR). That system uses multimodal features and is evaluated on a different target. We report CCC on E-DAIC for comparability but do not expect to match it with prosody alone. FairReFuse is evaluated on DAIC-WOZ, but the file quotes no DAIC-WOZ number for it.

Two ablations isolate our ingredients. The first replaces prompt-matched scores with classic symmetric, unmatched entrainment metrics computed over all turn pairs. The second removes the reluctance rule, fitting a full generalized additive model or a gradient-boosted model on the same features.

Beating a baseline means one of two things. On core-only turns, entrainment-plus-patient prosody must improve F1 or AUC over patient-only prosody, with a bootstrap 95% interval excluding zero and the improvement exceeding the permutation null. Alternatively, it must match the black-box model's core-only score while remaining exactly decomposable.

Why not just…?

Why not just use patient-only acoustic features and skip the dyad? This is the real threat, and the proposal is built to settle it rather than argue past it. The evidence that dyadic signals carry something single voices miss comes from therapeutic bond and marital affect (NED-PT; Lee et al.'s 76% session accuracy), not from depression. If prompt-matched entrainment adds nothing over patient-only prosody on core prompts, we will report that as the finding. It would still be a useful negative: clinicians would know that the "interactional" story, at least with a scripted agent, reduces to the participant's own voice.

Why not just drop the interviewer turns, as the text audit suggests? Removing Ellie's audio does not remove the script. The set of follow-up questions still determines how many participant turns exist, how long they are, and what topics they cover, all of which shape participant prosody. The full-versus-core-only gap on the patient-only model measures exactly this residual leak. If the gap is near zero, this simpler fix suffices for patient features, and our contribution narrows to the entrainment terms.

Why not just apply SHAP to a strong black-box audio model? Post-hoc attributions explain whatever the model learned, including shortcuts. Burdisso et al. found the shortcut only because their heatmaps happened to cluster on late script questions. SHAP on embedding dimensions also yields no clinically named quantity such as "pitch proximity toward the interviewer." We include the black box precisely to measure what transparency costs. If it beats our model by a wide margin on core-only turns, that is evidence the additive features miss real signal.

Why not just use classic symmetric Pearson or session-average entrainment metrics? They are transparent but cannot separate direction. With a fixed-voice interviewer they also conflate prompt identity with adaptation. The discriminative-control paper already found sham-contrasted local measures beat classic proximity, convergence and synchrony metrics on task efficiency (72% vs 64%). The unmatched-metrics ablation tests whether that advantage carries to depression, and in particular whether the unmatched version's apparent advantage collapses under the core-only restriction.

Proposed Experiments

The first decisive check is the shortcut test itself. We train the classic unmatched entrainment model under the full and core-only regimes on DAIC-WOZ using official splits plus nested cross-validation. If classic entrainment looks predictive in the full regime but drops to the permutation null in core-only, we have answered Burdisso et al.'s open acoustic question with a concrete mechanism. The prompts-only acoustic baseline quantifies the ceiling of that leak.

The second check is incremental value. Prompt-matched entrainment plus patient prosody is compared against patient prosody alone on core-only turns, using F1 and AUC with bootstrap intervals. We also report E-DAIC severity regression (CCC, RMSE) as a replication on a second partition.

The third check concerns interpretability and stability. For each admitted component we report its sign and shape, and its selection frequency across 100 bootstrap resamples. We then ask whether these agree with independent evidence. One test is whether the directional asymmetry of NED-PT reappears, with participant-toward-interviewer terms selected and reverse-direction terms not. Another is whether intensity terms enter nonlinearly, as the arousal ambiguity predicts. Components that flip sign across resamples are reported as unreliable, not explained.

The final check is a preregistered assumption test. If Ellie's per-prompt acoustic variance turns out not to be small, the fixed-interviewer identification argument fails. In that case we fall back to treating $a_{qd}$ as observed per instance and report the weaker, non-directional version of the analysis.