Site Signatures as Natural Backdoors in Psychiatric Brain-Imaging Classifiers

Brain-imaging classifiers for autism and depression learn from diagnostic labels that come from questionnaires and clinician interviews, and diagnostic practice differs from hospital to hospital. When one site diagnoses more liberally, or recruits patients and controls differently, a model can learn that site's scanner fingerprint as a stand-in for the diagnosis. Such a fingerprint acts much like a backdoor trigger, which is a hidden input pattern that forces a fixed output. Security research has a cheap test for this kind of trigger. Its premise is that predictions driven by a trigger stay locked when the input is heavily perturbed, while honest predictions wobble. We propose to point that test at clinical classifiers to flag the individual subjects whose diagnosis is being read off a shortcut instead of off brain biology.

The payoff is a per-subject reliability signal that needs no knowledge of what the confound is. Harmonization methods only remove confounds someone already thought to record, such as site. Our signal would also catch unrecorded ones, such as a protocol change partway through a study or a subgroup recruited differently. The same test doubles as a defense against a participating site that deliberately plants a trigger in a shared dataset. This threat is plausible in multi-site consortia, but no paper in the literature we reviewed has studied it.

Core Claim

Uncertainty-based detectors have found two opposite failure signatures. Adversarial inputs make an ensemble or dropout network suddenly uncertain, as shown by Dropout Inference in Bayesian Neural Networks with Alpha-divergences and Robustness to Adversarial Examples through an Ensemble of Specialists. Backdoored inputs show the opposite: they make the network abnormally certain and stable. STRIP detects this by blending an input with clean images and observing that trigger inputs keep their label, which gives low prediction entropy. PSBD detects it by showing that trigger inputs barely change their prediction when dropout is switched on at test time.

Our claim is that the second signature transfers to psychiatric imaging and has a natural target there. The best autism imaging models spread their predictive signal broadly across the cortex, according to Insights from an autism imaging biomarker challenge. A prediction that rests on genuine, diffuse biology should therefore be moderately unstable under perturbation. A prediction that rests on a compact site or protocol fingerprint correlated with how that site labels its patients should be over-stable, in the same way a trojaned input is. Rejecting or down-weighting the over-stable subjects should improve accuracy on held-out sites. It should do so more than rejecting the same number of subjects at random or by low softmax confidence.

This differs from the closest connecting work in two ways. The federated studies, Self-Driven Entropy Aggregation for Byzantine-Robust Heterogeneous Federated Learning, Mitigating malicious model fusion in federated learning via confidence-aware defense, and the dishonest-participant test in pF-GMCO on Rest-Meta-MDD, all screen model updates at the moment they are merged. We screen individual inputs at test time instead. None of those studies treats the subjectivity of the labels as the source of a backdoor-like shortcut. The adversarial-regularization work, such as DeepASD, aligns sites during training and never asks which individual predictions remain shortcut-driven afterward.

Research Program

We start from the pipeline of Deriving robust biomarkers from multi-site resting-state data. It defines brain regions with the data-driven Multi-Subject Dictionary Learning atlas, extracts a time series per region, and turns each subject's region-to-region connectivity matrix into a flat vector. The flattening uses the tangent-space embedding, which maps the curved space of connectivity matrices onto a flat space where averaging and blending are geometrically meaningful. That property is what makes the STRIP-style test well defined for connectomes.

For a subject's tangent vector $t$, we form blends $t_j = (1-\lambda)t + \lambda u_j$. Here $u_j$ ranges over $N$ held-out subjects drawn from sites other than the subject's own, and $\lambda \in (0,1)$ sets the blend strength. We score the subject by the entropy of the averaged prediction over those blends:

$$S(t) = -\sum_{c} \bar p_c \log \bar p_c, \qquad \bar p_c = \frac{1}{N}\sum_{j=1}^{N} p_\theta(c \mid t_j),$$

where $p_\theta(c \mid \cdot)$ is the classifier's probability for diagnosis $c$. A low $S$ means the diagnosis survives having most of the subject's connectome replaced by other people's, which marks the prediction as trigger-like.

We pair this black-box score, which works for the linear classifier of Abraham et al., with a PSBD-style score for a small dropout network trained on the same tangent vectors. The PSBD score is the variance of the predicted class across $K$ dropout passes. We train that network with the mass-covering alpha-divergence objective of Li and Gal, because ordinary dropout training underestimates uncertainty. Underestimation would blur exactly the contrast between stable and unstable predictions that we rely on.

The analysis has three parts. First, we run controlled injection on ABIDE. We add a low-amplitude synthetic signature to the connectomes of a subset of subjects. This signature is a fixed perturbation on a small set of connections and is not linked to any recorded site variable. We correlate it with the autism label only within chosen training sites, imitating a site that labels its own population differently. Because we planted it, we know which test subjects are shortcut-driven. Second, we repeat the controlled injection as a deliberate attack, with one site planting a trigger aimed at a demographic subgroup. This checks whether the same score handles the security case. Third, we apply the score with no injection to the real data on ABIDE and Rest-Meta-MDD and ask whether it finds naturally occurring shortcuts.

In passing, we will check a confusion raised by the electroencephalography (EEG) literature. Peddi et al. report that low signal entropy in the brain recordings themselves tracks the severity of post-traumatic stress symptoms. That is entropy of the input. Our score is entropy of the model's output under perturbation. We will check that the two do not correlate in our imaging data, so a flagged subject is not simply a severe patient.

Baselines

The naive reference point is to accept every prediction. Its reported level of performance is about 67% accuracy on ABIDE under inter-site validation, from Abraham et al.. The challenge winners in Bazeille et al. reach an AUC of about 0.80 internally and 0.72 on the external EU-AIMS cohort. On Rest-Meta-MDD, pF-GMCO reports 79.07% average accuracy. We reproduce the Abraham pipeline ourselves rather than relying on those figures, so that every method sees identical splits.

The second baseline rejects subjects by low softmax confidence or high single-pass predictive entropy. It is the cheapest abstention rule and isolates the key ingredient: whether perturbation-induced stability carries information beyond plain confidence.

The third baseline is the original detectors run unmodified on the tangent vectors. That means STRIP with naive blending, and PSBD with standard dropout instead of alpha-divergence training. For context, the reported numbers on images are 0% false acceptance and 0% false rejection in most CIFAR-10 cases for STRIP, and a true positive rate near 1.0 at a 10–15% false positive rate for PSBD. We found no reported result on Lacuna for either detector on neuroimaging or connectome data.

The strongest competitor is harmonization: removing site effects from the features while preserving diagnosis, then retraining. We also include a harmonized model combined with our score.

Every method gets the same leave-site-out folds, the same feature pipeline, and the same budget of hyperparameter search on inner folds. All abstention rules are compared at matched rejection rates. We count the proposal as beating a baseline only under two conditions. On injected data, it must detect shortcut-driven subjects with clearly higher AUROC than softmax confidence. On real data, it must give higher held-out-site AUC at equal coverage than both confidence-based and random rejection. A gain that disappears once harmonization is applied would count as a loss.

Why not just…?

Why not just harmonize the sites? This is the real threat. If the confound is recorded site, removing site effects may get most of the benefit with no new machinery. But the challenge results of Bazeille et al. still show a drop from 0.80 to 0.72 on an unseen cohort. In addition, harmonization that preserves diagnosis keeps any site-specific labeling bias, because that bias is carried by the diagnosis variable it protects. The controlled-injection experiment is designed so that the planted confound is invisible to site-based harmonization. If our score helps only when the confound is recorded site, we will report that harmonization suffices and the contribution narrows to the security case.

Why not just threshold the classifier's confidence? Deterministic networks are confidently wrong under shift, as the image results of Abbasi and Gagné and Li and Gal show. More importantly for us, a shortcut and genuine biology can yield equally confident predictions. They differ only in how the prediction responds to perturbation, which is the distinction PSBD exploits. The confidence baseline tests this directly. If confidence matches our score on detection AUROC, the stability idea adds nothing and we will say so.

Why not just audit the model by predicting site from its features? A site probe shows that site information is present. It does not show which individual predictions depend on that information, and it only works for confounds someone has labeled. We will run a counterfactual version that removes the site component per subject and measures the resulting prediction change. That version serves as an oracle for the recorded-site case, and the injected unrecorded confound tests what it cannot see.

Proposed Experiments

On ABIDE, under leave-site-out folds, we vary three things: injected-signature strength, the fraction of contaminated sites, and the blend strength $\lambda$. For each setting, we measure AUROC for identifying shortcut-driven test subjects, and accuracy-versus-coverage curves on clean held-out sites. We repeat the deliberate-attack variant with a subgroup-targeted trigger and report the true and false positive rates. These can be set beside the image-domain numbers of STRIP and PSBD.

On Rest-Meta-MDD, with 23 sites, and on uninjected ABIDE, we test the natural-shortcut hypothesis. Flagged subjects should cluster in sites whose patient-to-control ratio departs from the pooled ratio. Removing them from training should raise held-out-site AUC more than removing the same number at random or by low confidence.

We also measure the cost of the method on clean data: the share of correctly diagnosed subjects rejected at each operating point. The literature flags this as the chronic weakness of uncertainty-based rejection, and it decides whether the score is usable in a clinic.

The decisive check is the unrecorded-confound condition. If over-stability flags shortcut-driven subjects there and harmonization does not, the claim holds. If not, the idea reduces to harmonization with extra steps.