Auditing Hidden Branch Leaks in Multi-Stage Diagnostic AI

Many diagnostic AI systems that match specialists are not a single classifier. They are pipelines that first make a discrete decision and then act on it. A dementia model first decides whether a patient has normal cognition, mild impairment, or dementia, and only then asks what the cause is. A chest X-ray triage system first flags a film as abnormal and then routes it to a pathology classifier. Security researchers have a precise name for a risk that such branching creates: an implicit information flow. This is when a variable steers an outcome by changing which branch runs, without ever being copied into the computation that produces the answer. In clinical pipelines, the variable of concern is often something that should not matter, such as the scanner or hospital site that produced an image. It can shift the first-stage decision and so change the final diagnosis, while leaving no trace in the features the second stage reads.

This proposal claims that this branch-routed dependence is a measurable, separate, and largely unmonitored source of the external-validation drops seen in expert-level diagnostic models. It also claims that current audits are poorly suited to see it. Gradient-based saliency cannot see it at all, because the gradient through a hard threshold is zero almost everywhere. Per-stage SHAP attributions look at each stage's inputs and miss the gate that connects the stages. We borrow one tool from program analysis, forced-branch execution: rerun the program with a branch outcome fixed, as the fuzzer in Matryoshka does to expose implicit flows. We adapt it to clinical pipelines, where a rare advantage exists: the true intermediate label, such as clinician-assigned cognitive status, is recorded. We can therefore force the branch to its correct value and measure how much of the site dependence disappears.

Core Claim

Let a pipeline produce a final diagnosis $\hat y = g_{b(x)}(x)$. Here $x$ is the patient's inputs, $b(x)$ is the discrete first-stage decision, and $g_b$ is the second-stage model used on branch $b$. Let $s$ be a variable that should not influence the diagnosis once the true disease $y$ is known; our main choice is acquisition site. The spirit of noninterference, the property formalized for programs in Towards a Flow- and Path-Sensitive Information Flow Analysis, is that the conditional mutual information $I(s;\hat y \mid y)$ should be zero. We split this quantity by rerunning the pipeline with the branch forced to the recorded true stage-1 label $b^*$:

$$ L_{\text{total}} = I(s;\hat y \mid y), \qquad L_{\text{explicit}} = I(s; g_{b^*}(x) \mid y), \qquad L_{\text{implicit}} = L_{\text{total}} - L_{\text{explicit}}. $$

$L_{\text{explicit}}$ is the site dependence that survives when the gate is correct, meaning dependence carried through features. $L_{\text{implicit}}$ is the part routed through the gate. The difference is not guaranteed to be positive, because forcing can also expose dependence, and we will report it as a signed quantity.

We make two falsifiable predictions. First, on real clinical pipelines $L_{\text{implicit}}$ is a non-trivial fraction of $L_{\text{total}}$. Second, it predicts the drop in accuracy on an external cohort better than feature-level attributions do. We then test a targeted repair. We relax the gate into a smooth soft branch, in the spirit of Matryoshka's conversion of nested predicates into a continuous function. We then penalize the dependence of that soft branch on $s$ given $y$. The question is whether this closes the external gap more cheaply than debiasing the final output.

Following the path-sensitive analysis above, which lets security labels depend on which branch is taken, we also report leakage separately for each branch. A site dependence that exists only in the "dementia" branch but never changes the etiology call there is a false dependency, not a harm.

Research Program

The primary testbed reproduces the two-stage pipeline of Multimodal deep learning for Alzheimer's disease dementia assessment on NACC, with OASIS as the external cohort. That paper pairs a 3D-CNN on MRI with CatBoost on clinical variables, produces a cognitive-status score (DEMO) that gates an Alzheimer's-versus-other-dementia score (ALZ), and explains stages with SHAP. It motivates this work because its information truly passes through a discrete stage. It also supplies the pipeline we reuse. Access requires the cohorts' standard data-use agreements.

The sensitive variable $s$ is the NACC contributing center. We also add sex and age bin as secondary variables, to separate site shortcuts from demographic ones. Mutual information is estimated with a held-out classifier that predicts $s$ from $(\hat y, y)$. We compare this against a plug-in estimate over discrete outputs and report bootstrap intervals, since estimator choice is a known source of fragility.

A second testbed checks that the effect is not specific to dementia. We build a gated chest radiograph pipeline on ChestX-ray14. A DenseNet first decides normal versus abnormal, and a second DenseNet in the style of CheXNeXt then produces the 14 pathology labels. The file does not document which metadata ship with this dataset, so we will use whatever patient or acquisition variables the release provides as $s$. The oracle branch in this testbed is "any positive label."

The analysis stage computes $L_{\text{explicit}}$ and $L_{\text{implicit}}$ for each pipeline, both overall and per branch. We then correlate these quantities with the NACC-to-OASIS accuracy drop across many trained pipelines, which vary by random seed, backbone, and gate threshold. This gives a population of models rather than one point.

The repair stage trains the relaxed pipeline end to end. The gate becomes a softmax with temperature annealed toward hard decisions. The penalty is an adversary that tries to predict $s$ from the soft branch probabilities within each true class $y$. At test time the gate is hard again, so the deployed system keeps its interpretable staged structure.

Baselines

The naive reference point is the published two-stage pipeline trained as in the original paper, with no audit and no repair. On it we report the paper's own numbers: 4-way accuracy of 0.777 ± 0.011 on NACC and 0.730 ± 0.045 on OASIS, a gap of roughly 0.047 (Multimodal deep learning for Alzheimer's disease dementia assessment). For the chest pipeline, the reference is CheXNeXt-style performance, where the reported atelectasis AUC is 0.862 versus 0.808 for radiologists (CheXNeXt). No reported result exists on Lacuna for any site- or demographic-leakage metric on these benchmarks, so all leakage numbers will be new measurements.

The strongest audit competitor is per-stage SHAP, exactly as used in the dementia paper. We give it the same data and the same pipelines, and rank how much each model depends on site by the total SHAP mass on site-predictive features. A second audit competitor is gradient saliency in the style of Neutaint. Neutaint recovers implicit flows in programs by training a smooth neural surrogate and taking gradients of outputs with respect to inputs. We implement both variants: gradients through the real hard-gated pipeline, and gradients through a smooth surrogate fitted to it. The surrogate variant is the fairer and harder competitor, because smoothing is exactly what lets Neutaint see branch effects. Our audit beats these competitors if $L_{\text{implicit}}$ achieves a higher rank correlation with the external gap across the model population, with non-overlapping bootstrap intervals.

The strongest repair competitor is output-level adversarial debiasing. The same adversary is applied to the final prediction $\hat y$ instead of the gate, with the same adversary capacity, the same penalty-weight search budget (identical grid, selected on NACC validation only), and the same compute. We also compare against a single-stage end-to-end model with no gate, trained with the same budget. The branch-level repair counts as a win if it shrinks the OASIS gap by more than output-level debiasing does, while keeping NACC accuracy within one reported standard deviation of 0.777.

Two ablations isolate the key ingredient. The oracle-forced pipeline, which uses the true stage-1 label at test time, gives an upper bound on what fixing the gate alone could achieve. A relaxed pipeline trained without the penalty separates the effect of smoothing from the effect of the constraint.

Why not just…?

Why not just run SHAP on the whole pipeline as a black box? Perturbation-based SHAP over the full pipeline does capture total influence, including influence routed through the gate, so it is not blind. What it cannot do is tell you the route, and the route determines the fix. Feature harmonization addresses $L_{\text{explicit}}$, while gate constraints address $L_{\text{implicit}}$. We include whole-pipeline SHAP as an additional audit baseline. If its site attribution predicts the external gap as well as $L_{\text{implicit}}$ does, the audit half of our claim fails, and only the repair half remains to be tested.

Why not just do standard causal mediation analysis? This is a genuine threat to the novelty claim, and we accept it. Our decomposition is a mediation analysis in which the mediator is the pipeline's own discrete control decision, and the intervention is forcing that decision to a recorded clinical label. The contribution is not the estimator. It is three things together: identifying the branch as the mediator that practice ignores, noting that clinical pipelines supply a ground-truth value for this intervention, and showing empirically that this mediator matters for external validity. If $L_{\text{implicit}}$ turns out near zero across pipelines, the idea is refuted cleanly, and that is a useful negative result for the staged designs these papers favor.

Why not just train a single end-to-end model and avoid branches entirely? A model without explicit branches still has soft internal routing, and it gives up the staged outputs clinicians read, such as the separate DEMO and ALZ scores. The dementia paper uses exactly this staging to line up with neurologist workflow. The single-stage baseline tests whether this matters empirically. If it matches the repaired pipeline's external accuracy, we will report that the simplest route to robustness is to remove the gate. We will also report whether it pays for that robustness in leakage or interpretability.

Why not just debias the final output? Constraining $\hat y$ can drive $L_{\text{total}}$ down while the gate still depends on site. The dependence can be offset by the second stage on the training distribution but not on a new cohort. That offsetting is precisely what we expect to break under shift. The output-debiasing baseline, run with an identical budget, is where this prediction is tested directly.

Decisive Checks

The idea stands or falls on three measurements. The first is whether $L_{\text{implicit}}$ exceeds its bootstrap noise floor on NACC for the reproduced pipeline. The second is whether, across at least thirty trained pipelines, $L_{\text{implicit}}$ correlates with the NACC-to-OASIS accuracy drop more strongly than per-stage SHAP, whole-pipeline SHAP, or surrogate-gradient saliency. The third is whether the gate-level penalty narrows the roughly 0.047 external gap more than output-level debiasing, without losing more than 0.011 of NACC accuracy. A replication of the first two measurements on the gated ChestX-ray14 pipeline would show that the phenomenon is a property of staged diagnosis rather than of dementia cohorts.