Machine-learning models for depression and other psychiatric conditions are trained to reproduce labels that clinicians produce from interviews and rating scales. A model that scores well against those labels on historical data may therefore be learning what the clinician already knows. If so, it will look accurate in retrospective evaluation and add nothing once a psychiatrist is in the room. We propose to measure this directly, before any live deployment. We treat the clinician's own prediction and the model's prediction as two sources of evidence about the same patient outcome. We then split the information they carry into four parts: what both share, what only the clinician has, what only the model has, and what appears only when the two are combined. The split uses partial information decomposition (PID), an information-theoretic method for dividing what two sources jointly say about a target into these four parts.
The claim is that standalone accuracy mostly measures the shared part, which is clinically worthless. Only the model-only and combined-only parts can produce a gain when the model sits beside a clinician. These two parts also differ in how hard they are for a human to use. We expect psychiatrists to absorb model-only evidence fairly well. We expect them to largely fail to absorb combined-only evidence, especially when their known distrust of algorithms is in play. We test this on the EMBARC antidepressant trial. Psychiatrists make predictions first without, then with, a biomarker model's output. We ask whether the decomposition, computed in advance, predicts how much the human-plus-model team actually gains.
Core Claim
Let $H$ be the clinician's prediction of treatment response from routine clinical information, $A$ the model's prediction from EEG or fMRI, and $Y$ the observed response. PID writes
$$I(H, A; Y) = R + U_H + U_A + S,$$
where $R$ is redundant information (available from either source), $U_H$ and $U_A$ are information unique to the clinician and to the model, and $S$ is synergistic information (available only from the pair jointly). The model's standalone mutual information $I(A;Y) = R + U_A$ is what retrospective AUC tracks. The ceiling on what the model can add to a clinician is the conditional information
$$I(A; Y \mid H) = U_A + S.$$
Neither quantity says how much of that ceiling a real clinician captures. We therefore define the team's extraction efficiency from the clinician's revised prediction $Z$, made after seeing $A$:
$$\eta = \frac{I(Z; Y) - I(H; Y)}{U_A + S}.$$
The specific, falsifiable claim has two parts. First, $R$ dominates $I(A;Y)$ for current psychiatric biomarker models, so standalone accuracy overstates their value. Second, realized gain is better predicted by $U_A$ and $S$ entered separately than by their sum, because clinicians extract unique evidence much more efficiently than synergistic evidence.
If the second part fails, the honest conclusion is that ordinary conditional information is enough and the full decomposition is unnecessary. We treat that as a reportable outcome, not a failure to argue around.
Why This Intersection, and What Is Already Done
Two lines of PID work in medicine motivate the measurement but stop short of the clinician.
ICYM²I decomposes information between ECG and chest X-ray for structural heart disease, with inverse-probability weighting to undo the bias created when tests are ordered on clinical suspicion. Correcting that bias shrinks the X-ray's apparent unique information from about 5% to 1.8% and raises redundancy from 0.10 to 0.48. That is exactly the "looks informative retrospectively, isn't" pattern we target. Its two sources are both machine inputs, however, and its condition has an objective reference standard.
I2MoE routes health records through unique, redundant and synergistic experts. Its ablations show the synergy and redundancy experts barely move AUROC (0.7714 versus 0.7627 for XGBoost). This constrains us: decomposition as an architecture rarely pays off. We therefore use PID as a measurement instrument, not as a model component.
Luppi et al.'s ADNI analysis uses a dynamic extension of PID on brain signals as a disease marker. That is a different use: decomposing within the brain rather than between clinician and model.
On the human side, Maslej et al. show in a randomized study of 83 psychiatrists that identical treatment recommendations are rated 3.61 when attributed to a human and 3.13 when attributed to AI. They measure perception on Likert scales, not information transfer to outcomes. Horng et al. and Kottlors et al. argue, from sepsis triage and radiology respectively, that retrospective accuracy cannot reveal whether decisions change.
The candidate gaps in the literature include a "PID of the clinician-AI interface." To our knowledge no paper has computed it, and certainly none has tested whether it predicts realized team gain. That test is the contribution. The obvious alternative was to apply bias-corrected PID to fMRI plus EEG in EMBARC. We rejected it as the core idea: in a trial where everyone receives both modalities, there is little curation bias to correct, and the result would not touch the human-in-the-loop gap.
Research Program
Setting. We use EMBARC, a randomized trial of sertraline versus placebo with pre-treatment resting EEG, fMRI and clinical assessments, and week-8 HAMD-17 outcomes. The trial has three properties we need. Biomarkers were acquired for everyone, so the "missingness as signal" trap identified by ICYM²I does not apply. Outcomes were scored by trial raters, not by the psychiatrists making predictions, so $Y$ is not a copy of $H$. And randomization lets the target be response under sertraline specifically.
The outcome is still a subjective scale. That is the field's ground-truth problem, and we do not claim to escape it. We claim only that $H$ and $Y$ come from different people at different times.
Model sources ($A$). We reproduce two published models. The first is the EEG causal-forest policy of Learning Optimal Biomarker-Guided Treatment Policy, whose central biomarker is frontal-central eyes-closed theta power. The second is the fMRI-plus-EEG graph network of Deep graph learning of multimodal brain networks. We add a third model trained explicitly to predict $Y$ from the residual of a model of $H$, as described below.
Clinician source ($H$). We recruit about 60 psychiatrists, roughly matching Maslej et al.'s sample. Each reads about 40 de-identified baseline clinical packets drawn from the 309 EMBARC participants. A packet contains demographics, history, baseline HAMD items and comorbidities, but no biomarkers. Each psychiatrist gives a probability of sertraline response. Every patient is read by several clinicians, so we can compute PID both per clinician and on a consensus.
Team source ($Z$). After the first prediction, the psychiatrist sees the model's output and revises. The display condition is randomized between psychiatrists, crossing two factors. The first is source attribution (AI versus "a colleague's biomarker read"), following Maslej's manipulation. The second is format: the model's standalone probability, or a conditioned display stating how the model's view should shift a clinician holding the psychiatrist's own prior. The conditioned display is designed to make synergistic information usable.
Estimation. We discretize $H$, $A$ and $Z$ into three to four probability bins and binarize $Y$ as at least 50% HAMD reduction. We compute the decomposition with the convex-optimization (BROJA-style) definition of Optimizing Bivariate PID. Because definitions of redundancy disagree and known inconsistencies exist, we also report a minimum-mutual-information redundancy. A conclusion counts only if it holds under both definitions.
Sample size is small, so all quantities carry patient-level bootstrap confidence intervals with a bias correction checked on shuffled-$Y$ nulls. We deliberately avoid the continuous nearest-neighbour estimators, which need far more data than EMBARC provides.
We also test a secondary hypothesis. The digital-phenotyping finding that 72% of depression-score variance lies between people (Ikäheimonen et al.) suggests the biomarker's redundancy with the clinician is concentrated in baseline severity and stable traits. We check this by recomputing the decomposition within strata of baseline HAMD.
Baselines
The naive reference point is the clinician alone, $I(H;Y)$ and its AUC. Any claim of team benefit must beat this, and $\eta$ is normalized against it.
The strongest existing predictive method is the multimodal graph network. It reports $R^2 = 0.24$ for sertraline symptom change (Deep graph learning of multimodal brain networks). The EEG policy tree reports a 17.4% improvement in response rate (Learning Optimal Biomarker-Guided Treatment Policy). We retrain both on identical cross-validation folds, with patients assigned to folds before any clinician reads them. Each gets the same hyperparameter budget. Predictions are always out of fold.
These models are not competitors to beat on accuracy. They are the sources whose standalone accuracy we argue is misleading. The first claim succeeds if, for at least one of them, $R$ exceeds half of $I(A;Y)$ with confidence intervals excluding the shuffled null.
For the core question, which pre-deployment quantity best predicts realized team gain, the strongest competitor is incremental value. Concretely, this is the improvement in log-loss or AUC from adding $A$ to a logistic model of $H$, which is an empirical estimate of $U_A + S$. No reported result found on Lacuna for clinician-conditional incremental value of any psychiatric biomarker, nor for psychiatrists' unaided accuracy at predicting EMBARC response.
We also include standalone AUC as the conventional predictor, and the attribution baseline described in the Why-not section below. The comparison is a regression across the clinician-by-model-by-display cells of realized gain $I(Z;Y) - I(H;Y)$, scored by held-out explained variance under leave-one-clinician-out cross-validation. The decomposition "beats" incremental value only if entering $U_A$ and $S$ separately improves held-out fit, with the gain's confidence interval excluding zero.
The ablation that isolates the key ingredient is the display manipulation. If synergistic information is what clinicians fail to extract, the conditioned display should raise $\eta$ mainly in cells where $S$ is large relative to $U_A$. That is an interaction effect, not a main effect. A uniform rise in $\eta$ across all cells would mean the display simply helps, and would not support the decomposition.
Why not just…?
Why not just compute incremental AUC of the model over the clinician? This is the real threat. It already estimates $U_A + S$, the total the model can add. Our proposal keeps it as the main baseline and asks one narrow question: does splitting that total into unique and synergistic parts predict human uptake better? Evidence that it might comes from the decision literature on automation: a standalone signal the clinician can adopt wholesale differs from one that only helps through careful integration. But nothing in the file measures this. If the leave-one-clinician-out comparison shows no gain from the split, we will report that conditional information suffices, and recommend it as the cheaper pre-deployment screen.
Why not just use SHAP attributions on the biomarker model? SHAP-style attributions, like the 13-feature subset in the REST-meta-MDD stacking study, explain which inputs drive the model. They say nothing about overlap with what the psychiatrist already knows. A feature with large attribution can be entirely redundant with baseline severity, which the clinician reads off the chart. We include SHAP-weighted feature importance as a predictor in the team-gain regression to confirm it carries no extra signal.
Why not just run a live randomized deployment? That is the definitive test, and EMBARC predictions cannot substitute for it. But such trials are expensive, and the direction overview identifies this cost as the main reason they are rare. Our study is prospective in clinician reasoning but retrospective in patients. It provides a go/no-go criterion: a model whose $U_A + S$ is near zero, or whose $\eta$ stays near zero under every display, should not reach a live trial. Clinicians changing their routines over time is out of scope here and is acknowledged as a limitation.
Why not just train the model to be complementary to the clinician? This is compatible with the proposal, so we include it: the third model is fit on residuals from a model of $H$. If it maximizes $U_A$ while lowering $R$, it is a direct test of the first claim. Standalone AUC should fall while team gain rises. That dissociation would be the clearest demonstration that retrospective accuracy is the wrong target.
Decisive Checks
The work stands or falls on three results. First, redundancy with the clinician accounts for most of each published biomarker's standalone information, under both redundancy definitions. Second, separating unique and synergistic information predicts held-out team gain better than conditional information alone. Third, the conditioned display raises extraction efficiency specifically where synergy is large, and AI attribution lowers extraction of unique information, linking Maslej et al.'s aversion effect to lost outcome information rather than lower ratings. Failure of the second check while the first holds still delivers a useful result: a validated, cheap screen for which psychiatric biomarker models deserve prospective trials.
References
- ICYM$^2$I: The illusion of multimodal informativeness under missingness
- Understanding Structured Health Data through Interaction-Aware Mixture-of-Experts
- Synergistic and redundant information dynamics are modulated by Alzheimer's disease and cognitive impairment
- Out with AI, in with the psychiatrist: a preference for human-derived clinical decision support in depression care
- Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning
- AI support for accurate and fast radiological diagnosis of COVID-19: an international multicenter, multivendor CT study
- Learning Optimal Biomarker-Guided Treatment Policy for Chronic Disorders
- Deep graph learning of multimodal brain networks defines treatment-predictive signatures in major depression
- Optimizing Bivariate Partial Information Decomposition
- Novel Inconsistency Results for Partial Information Decomposition
- Variability in self-reported depression symptomology and associated behavioral markers in digital phenotyping
- Multi-feature concatenation and multi-classifier stacking: an interpretable and generalizable machine learning method for MDD discrimination with rsfMRI