Whose Scale Is the Truth? Deferring to Clinicians When the Label Is Also Subjective

Several recent systems let a machine learning model and a clinician share decisions. The most careful of them, Preferential Mixture-of-Experts, makes the clinician's rule the default. The model takes over only for patients where it provably does better. That design has a hidden assumption that breaks in psychiatry. To show that the model "does better," you need an outcome label. In depression, that label is itself a symptom rating scale: a questionnaire filled in by the patient, or an interview score assigned by a clinician. Often it is the same instrument the clinician's rule reads from. The rule and the answer key then share the same rater, the same reporting biases and the same blind spots. The rule will look more reliable than it is, and the model will be allowed to overrule it in the wrong places.

We propose to measure and correct this circularity. The testbed is a depression trial that recorded two independent raters of the same construct at each visit. STAR*D collected a clinician-rated scale (QIDS-C16) and a patient self-report version (QIDS-SR16) side by side. That lets us train and audit a clinician-preferring deferral system three ways: against the clinician's own instrument, against the patient's instrument, and against a consensus of both. Our claim is that the choice of answer key changes three things: how often the system follows the clinician, which patients it overrules, and whether those overrules are correct. We further claim that a gate told explicitly how much the patient and clinician disagree will overrule more accurately than one that is not. The deliverable is a deferral system whose record of when and why it departed from the clinician holds up under an independent rater. We also deliver a measured estimate of how much same-instrument evaluation inflates apparent trust in clinical rules.

Core Claim

In subjective-diagnosis settings, the measured value of deferring to a clinician rule is not a property of the rule alone. It is a property of the rule paired with the instrument used to score it. Suppose a clinician-preferring gating model is trained with its safety constraint scored on the same clinician-rated scale that the rule reads. We predict three effects compared with training on an independent or consensus outcome. First, it will show higher human-rule coverage, meaning the fraction of patients handled by the rule. Second, it will choose a different set of patients to overrule. Third, a larger share of its overrules will be wrong when checked against the patient's own report. Adding the patient–clinician rating gap as an input to the gate should concentrate overrules on patients whose two raters disagree. Examples are patients who under-report, or patients whose physical illness inflates somatic items. It should also make the system's explanation of each overrule more stable across answer keys.

The mechanism reuses Pradier et al.'s formulation, with one quantity made explicit. Let $g(x)\in[0,1]$ be the gate's probability of following the human rule for a patient with features $x$. Let $L_Y$ be the prediction loss measured against outcome label $Y$. Let $\epsilon$ be the tolerated loss margin. The gate solves

$$\max_g \; \mathbb{E}[g(x)] \quad \text{subject to} \quad L_Y(\text{combined system}) \le L_Y(\text{unconstrained model}) + \epsilon .$$

The original paper treats $Y$ as given. Our entire contribution concerns what happens when $Y$ is a rating that shares its rater with the rule. We compare $Y_{\text{clin}}$ (QIDS-C remission), $Y_{\text{self}}$ (QIDS-SR remission) and $Y_{\text{cons}}$ (remission on a latent severity estimated jointly from both scales).

Why This Gap and Not the Obvious One

The obvious move at this intersection is Gap 1 in the research file. It would build a gate that arbitrates between a patient's self-report and an objective brain biomarker, adding EEG or fMRI to Preferential MoE. We deliberately do not lead with it, for two reasons.

The first reason is evidence from Predictive modeling of treatment resistant depression using data from STAR*D and an independent clinical study. That paper builds treatment-resistance models from questionnaires and demographics alone. It reports AUC 0.70–0.78 in STAR*D and 0.72–0.77 in the independent RIS-INT-93 cohort, and concludes that biological markers are "not strictly necessary for early prediction." A biomarker gate would have to beat that cheap baseline before it could say anything about arbitration.

The second reason is that the biomarker datasets in the file cannot support the arbitration question. Deep graph learning of multimodal brain networks predicts sertraline and placebo response in EMBARC from fMRI and EEG, with R² of 0.24 and 0.20. Its target is a single clinician scale (HAMD17). No independent rater is recorded in the file, so a "subjective versus objective" disagreement can never be scored against ground truth. Sun et al. shows that EEG under self-referential punishment helps classify suicide ideation (AUC 0.92 multimodal vs 0.80 for depression classification), but with 77 participants. Both are motivating evidence that subjective instruments miss things. Neither gives the paired measurements needed to test a deferral rule honestly.

The circularity problem, by contrast, can be tested now. STAR*D provides paired raters at scale. The problem also sits upstream of every biomarker proposal, since any biomarker gate will face the same answer-key question.

The contrast with the closest work is concrete. Preferential MoE supplies the method: a default-to-human gate under an error guarantee with a linear, auditable gating function. It was evaluated on HIV therapy and MDD management with fixed labels, and it assumes the human rule is a crisp heuristic scored against a trusted outcome. We keep the method and change only the evaluation target and one gate input. We then ask whether its conclusions survive. AICare motivates the human side of the work. Its 16-clinician study found that interpretability is double-edged: exposed illogical reasoning collapsed trust. It also found that junior clinicians used explanations as a scaffold while seniors adversarially verified them. Its domains were nephrology and obstetrics, where labels come from laboratory values and are not contested. MLlm-DR and BRI-MH produce explained depression scores but train and evaluate against a single questionnaire. They have no mechanism for the clinician to be overruled or to overrule. The DAIC-WOZ validity study is a constraining result. It shows that a model reading only the interviewer's prompts reaches F1 0.88, so models on subjective data readily learn the evaluator rather than the patient. Our shared-instrument hypothesis is the tabular, longitudinal analogue of that finding.

Research Program

The work has three parts.

Part 1: build the rule, the model and the gate. The human rule is a measurement-based-care rule of the kind clinicians actually apply. It predicts non-remission if the clinician-rated QIDS-C16 has improved by less than a fixed fraction by week 2, reflecting the early-trajectory signal that the STAR*D modeling paper found most predictive. The ML expert is a gradient-boosted classifier over the same socio-demographic, comorbidity, WSAS, PCS and early-visit features that paper used. The gate is linear, as in Preferential MoE, so each overrule can be traced to named covariates.

We train the gate under each of the three answer keys. The consensus key comes from a simple two-rater latent severity model: one latent severity per visit, with separate loadings and noise terms for QIDS-C16 and QIDS-SR16. We fit it on training folds only. In a second variant, the gate also receives the baseline and week-2 rating gap (self-report minus clinician rating, standardized), and its sign.

Part 2: analysis. We quantify circularity directly. For each answer key we report human-rule coverage, system AUC, the overlap between overrule sets across keys (Jaccard index), and overrule precision. Overrule precision is the fraction of overruled patients for whom the ML expert was right and the rule wrong. We always score it on a key other than the one used in training. We also characterize who gets overruled under each key, using the linear gate's coefficients and a subgroup breakdown by anxiety, chronicity and physical-health burden. These are the subgroups the STAR*D paper flags, and plausibly the ones where somatic and self-report distortions differ between raters.

Part 3: a small clinician reader study, in the spirit of AICare. Twelve to twenty psychiatrists review de-identified STAR*D case vignettes. Some show the gate's overrule plus its audit trail. Others show the ML prediction with SHAP attributions. We record whether clinicians accept correct overrules and reject incorrect ones, with correctness scored on the held-out key, split by clinician seniority. This tests whether an explanation phrased as "the patient's and your ratings diverge by X, and patients like this remit differently" is used more appropriately than raw feature attributions. It speaks directly to AICare's scaffold-versus-adversarial-verification finding in a domain with no laboratory reference standard.

Baselines

The naive reference point is the human rule alone: the week-2 QIDS-C improvement threshold applied to every patient, scored on all three keys. Any system that cannot beat it on the independent keys has no business overruling a clinician. We run it on exactly the same test folds as everything else.

The strongest predictive competitor is the questionnaire-only model of the STAR*D treatment-resistance study. That paper reports AUC 0.70–0.78 on the STAR*D test set, 0.77 with a 30-feature model, and 0.72–0.77 on RIS-INT-93. We reimplement it on our splits, with its feature-selection procedure and an identical hyperparameter search budget. It is the strongest competitor because it is cheap, replicated and uses the same data. The gate does not need to beat its AUC outright. The gate succeeds if it stays within ε of this model's AUC on the consensus key while following the clinician rule for a substantial share of patients, and if its overrules show higher precision on the independent key than the competitor's disagreements with the rule do.

The strongest existing deferral method is Preferential MoE trained exactly as published, against the same-instrument clinician label. On HIV it reports AUC 0.74 against 0.69 for a standard mixture-of-experts, with human-rule coverage rising from 52% to 63%. The file gives no reported number for its MDD task, so we treat our reimplementation as the reference. Our version counts as beating it if its overrules are more precise on the patient-reported key, and not merely if it shows higher same-key AUC. A standard mixture-of-experts, in which human and model compete symmetrically, is included as the design that Preferential MoE itself improves on.

Two ablations isolate the proposal's ingredients. The first uses the consensus answer key with no rating-gap feature. It tests whether fixing the target alone suffices. The second uses the rating-gap feature with the same-instrument key. It tests whether the feature helps even when the target is circular. All variants share the ε grid, number of random restarts and folds. Restarts matter because the file notes that this constrained, game-style optimization is sensitive to initialization.

Why not just…?

Why not just train on the self-report label instead of the clinician label? Swapping which rater is treated as truth moves the circularity rather than removing it. The self-report scale carries its own distortions: stigma, concealment, poor insight. Sun et al. is motivating evidence that self-report misses high-risk states. We nonetheless run this as one of the three keys. If a gate trained on QIDS-SR alone matches the consensus-trained gate on overrule precision under both single-rater keys, then the consensus model is unnecessary and we will say so.

Why not just average the two scales, or drop patients where they disagree? Averaging is the real threat here, because it is nearly what the consensus model does. If a plain average gives the same overrule set, then the latent model adds nothing and the finding reduces to "use both raters." That would still be a useful and publishable correction to deferral evaluation. We include the simple average as an extra comparison. Dropping disagreeing patients is worse. It removes exactly the patients where deferral matters most, and our subgroup analysis will show how large that group is.

Why not just use the questionnaire model with SHAP and let the clinician decide? This is the status quo that BRI-MH proposes for mental health. It gives no default to the clinician and no guarantee on when the model's view should prevail. AICare documents visual overload and trust collapse when raw attributions are shown. Part 3 of the program tests this alternative head to head. If psychiatrists' decision quality is no worse with SHAP than with the gate's audit trail, the interface claim fails. The circularity finding would stand independently.

Why not just add EEG or fMRI biomarkers so the label problem goes away? Biomarkers do not remove the label problem. The EMBARC models in Nie et al. are trained against HAMD17, itself a clinician rating, so a biomarker expert inherits whatever answer-key bias that scale carries. As a secondary analysis, we train the same-instrument gate on EMBARC with an EEG/fMRI expert. We ask whether its overrules concentrate in the patients where the HAMD-based rule errs. We state plainly that EMBARC cannot test cross-rater correctness, because no second rater is available in the file.

Proposed Experiments and Decisive Checks

The primary dataset is STAR*D, using paired QIDS-C16 and QIDS-SR16 visits plus WSAS and PCS. We use patient-level train/test splits and report every metric on all three answer keys. External validation is on RIS-INT-93, whose outcome is HAMD17 remission (≤7) after six weeks of citalopram. That is a third, independent clinician instrument. It is available only if access can be obtained, since the file does not describe its release status. EMBARC serves the secondary biomarker analysis described above.

Metrics are AUC and calibration for the combined system, human-rule coverage, overrule precision scored on a held-out key, and cross-key Jaccard overlap of overrule sets. In the reader study we measure appropriate acceptance and rejection of overrules by clinician seniority.

The decisive check for the core claim is the gap between same-key and cross-key performance. Suppose the same-instrument gate's coverage and overrule precision drop by no more than their confidence intervals when rescored on QIDS-SR or on RIS-INT-93's HAMD17. Then shared-rater circularity is not practically important in this setting, and the proposal's premise is falsified.

The second check concerns the rating-gap feature. It should raise overrule precision on the held-out key and concentrate overrules among patients with large rater disagreement. If it does neither, the gate needs no disagreement awareness.

The third check is the reader study. Clinicians shown the audit trail should accept correct overrules and reject incorrect ones at a higher rate than clinicians shown SHAP. The effect should be clearest in junior clinicians, who in AICare were most prone to treating explanations as confirmation.