Tools that make brain-imaging analysis accessible to clinicians take one decision out of their hands: which measurements of brain activity to feed a diagnostic classifier. They do this with point-and-click dashboards, automated model search, and language-model agents. Resting-state fMRI offers many such measurements, each capturing something different. One measures signal strength in a region, one measures how well neighboring voxels agree, one measures how mirrored regions in the two hemispheres track each other, and one measures how connected a region is to the rest of the brain. Combining them improves accuracy. But an automated tool that picks the combination with the highest validation score is also picking the combination that best encodes demographic shortcuts such as sex, age, and scanner site. In autism cohorts, where girls are a small minority, that shortcut would show up as a model that under-diagnoses autistic girls more than the data imbalance alone would predict.
We propose to measure this effect directly and to fix it at the point where it happens. The measurement is "bias amplification": disparity in the model's predictions beyond the disparity already in the labels. The claim is that the metric-and-pipeline selection step inside accessible tools is itself a source of bias amplification in brain-imaging diagnosis. We further claim that the amplification concentrates in measurements that carry local signal variance, because variance differs by sex, age, and site. We then test whether a selection rule that scores pipelines jointly on accuracy and on multi-attribute directional amplification removes most of this effect at a small accuracy cost. We also test whether a much simpler fix does just as well.
Core Claim
Three bodies of work each stop just short of this question.
The first is neuroimaging work on population diversity, which motivates the proposal. The Cost of Untracked Diversity in Brain-Imaging Prediction and Population heterogeneity in clinical cohorts affects the predictive accuracy of brain imaging show that prediction accuracy falls as the demographic gap between training and test sets grows (r = −0.671 on ABIDE). They also show that Default Mode Network weights shift with cohort composition. Both papers measure lost accuracy and unstable weights; neither asks whether predictions exaggerate demographic skew. Towards robust and replicable sex differences in the intrinsic brain function of autism finds a replicable sex-by-diagnosis interaction in one metric, cross-hemispheric synchrony in dorsolateral occipital cortex. That result is descriptive and involves no classifier.
The second is the bias-amplification literature, which supplies the measurement. Directional Bias Amplification separates two directions: whether an attribute drives the task prediction, or whether the task drives the attribute prediction. Men Also Do Laundry: Multi-Attribute Bias Amplification shows that single-attribute scores hide bias through cancellation. On imSitu, the signed score was 0.7 while the absolute-difference score was 11.9. That paper also shows that fixes for single attributes can raise bias at attribute intersections.
The third supplies the mechanism we test. AutoML for NLP argues that search driven by aggregate accuracy selects shortcut-exploiting configurations. Why Fair Labels Can Yield Unfair Predictions proves that loss-minimizing predictors reconstruct a hidden sensitive attribute from proxy features. Disambiguating brain functional connectivity shows that changes in correlation between regions are often explained by changes in local variance rather than by changes in coupling. Put together, these suggest that variance-bearing metrics, such as fractional amplitude of low-frequency fluctuations (fALFF), are ready-made demographic proxies, and that an accuracy-driven selector will find them.
The closest tool-side work treats "fairness" in a different sense. CellAgent blinds its evaluator agent to tool names so that it does not favor popular methods. That addresses bias about tools, not bias about patients, and it covers single-cell data, not brain imaging.
The amplification measure matters because it separates harm from biology. A model that correctly uses a real sex-specific signal does not amplify; it matches the true co-occurrence of sex and diagnosis. A model that uses sex as a shortcut pushes predicted co-occurrence past the true rate. Following the form of the directional measure in Directional Bias Amplification, for a demographic cell $g$ and diagnosis $t$ let
$$\Delta_{g,t} = P(\hat{T}=t \mid G=g) - P(T=t \mid G=g),$$
where $\hat{T}$ is the predicted diagnosis and $T$ the true one. Amplification averages $\Delta_{g,t}$ with its sign aligned to the direction of the true association. Following Men Also Do Laundry, we take absolute differences over intersectional cells $g$ (sex × age band × site) so that opposing biases cannot cancel.
Research Program
We build an explicit pipeline space of the kind an accessible tool searches implicitly. It crosses three choices:
- Metrics: all 31 non-empty subsets of the five metrics used by Floris et al. These are posterior cingulate connectivity, cross-hemispheric synchrony, regional homogeneity, degree centrality, and fALFF.
- Denoising: their three pipelines, CompCor, global signal regression, and ICA-AROMA.
- Classifiers: an L2-regularized logistic regression, matching the diversity papers, and the population graph convolutional network of Spectral Graph Convolutions for Population-based Disease Prediction.
For every pipeline we record held-out AUC alongside single-attribute and multi-attribute directional amplification. Each quantity gets a bootstrap interval, and intersectional cells below a minimum size are pooled or dropped. The first analysis asks whether the pipeline chosen by accuracy sits in the upper tail of the amplification distribution, compared with a randomly chosen pipeline of similar accuracy.
The second analysis attributes amplification to individual metrics. We use leave-one-metric-out differences and test how well each metric decodes demographics, following the representation-leakage idea in Wang et al.. A mechanistic ablation then tests whether variance is the carrier. Each variance-bearing metric is replaced by a version residualized against regional signal variance, which removes the additive-variance component identified by Cole et al.
The intervention is a selection rule:
$$\pi^\ast = \arg\max_{\pi}\; \mathrm{AUC}(\pi) - \lambda\, \mathrm{MultiAmp}(\pi).$$
Here $\pi$ is a pipeline, both terms are computed on an inner validation fold, and $\lambda$ is fixed before touching the test folds. We choose $\lambda$ as the value that keeps inner-fold AUC within one point of the accuracy-only choice. We pair the rule with attribute-conditioned training, meaning demographics are supplied explicitly as inputs. Ashurst et al. predict that this removes the incentive to reconstruct demographics from proxies.
The rule is packaged as an audit step that any automated selector must pass. It could sit inside a GUI default, an AutoML loop, or the evaluator of an agent such as CellAgent. The step also shows the user a per-metric amplification readout. The accessibility contribution is this interface to the selection decision, not a new dashboard.
Baselines
Naive reference. The do-nothing point is a single fixed pipeline: posterior cingulate connectivity with CompCor and logistic regression. It represents a careful analyst who never searches. Its amplification tells us how much bias the data and model produce without any selection.
Accuracy-only selection. This is the default behavior of accessible automation. It uses the same pipeline space, the same nested cross-validation folds, and the same compute budget, but selects on AUC alone.
Strongest published classifier. This is the population graph network of Parisot et al., which reports 69.5% ASD accuracy on ABIDE versus a prior best of 66.8%. It is the strongest competitor on accuracy, and it builds its population graph partly from sex and site, which makes it a natural candidate for amplification. For bias amplification on ABIDE we found no reported result on Lacuna, so all amplification baselines will be measured by us under identical folds.
Post-hoc group-wise thresholds. Per-group decision thresholds are fit on the inner fold for the accuracy-selected pipeline. Wang and Russakovsky report that threshold tuning cut COCO amplification from 0.040 to 0.001, so this is the strongest cheap mitigation.
Lagrangian fairness constraint. Equalized-odds constraints are imposed during training, following FNNC, on the accuracy-selected pipeline and tuned with the same budget.
Harmonization. ComBat is applied before selection, the neuroimaging default.
Ablations. Two ablations isolate the key ingredient. Selection with a single-attribute penalty in place of the multi-attribute penalty tests whether intersections matter. The variance-residualized metrics test whether variance is what carries the bias.
What counts as beating the baselines. The proposal succeeds if amplification-aware selection lowers multi-attribute amplification relative to accuracy-only selection, with a bootstrap interval excluding zero, while losing no more than two AUC points. It must also match or beat thresholds and the constraint on multi-attribute amplification without raising single-attribute amplification.
Why not just…?
Why not just drop sex and age from the inputs? Ashurst et al. prove that unawareness incentivizes proxy reconstruction and can increase introduced unfairness. Wang and Russakovsky's masking paradox shows the same behavior empirically. We run unaware and attribute-conditioned versions of every pipeline, so the prediction is tested directly on brain data rather than assumed.
Why not just harmonize with ComBat or regress out covariates? The diversity papers found that neither linear regression nor ComBat stopped demographic heterogeneity from degrading accuracy or destabilizing network weights. That suggests the effects are not simple additive shifts. We still include ComBat as a baseline, because amplification is a different quantity from accuracy loss and could behave differently.
Why not just recalibrate thresholds per group after training? This is the real threat. The 0.040-to-0.001 COCO result shows that calibration can erase most measured amplification. The counter-evidence is Zhao et al.'s finding that a single-attribute fix raised multi-attribute bias on COCO from 14.5 to 17.2. Thresholds per sex may likewise hurt cells defined by sex, age, and site together. If intersection-level thresholds match our rule on both amplification measures, we will report that the fix belongs after selection, not inside it. That would still be a useful default for accessible tools.
Why not just skip automated selection and use one standard metric? This is also a live possibility. If the fixed single-metric pipeline has much lower amplification and similar AUC, the honest conclusion is that automated metric search costs fairness for little gain. The naive-reference baseline is there to detect exactly that outcome. Cole et al.'s ambiguity result and the accuracy gains reported for combined metrics suggest a real accuracy gap, but on ABIDE that gap must be measured, not assumed.
Proposed Experiments
Primary cohort. The primary study uses ABIDE I and II, with intersectional cells over sex, age tercile, and site. Selection is nested inside site-stratified cross-validation, and a leave-site-out variant checks transfer. The design is then replicated on the Healthy Brain Network cohort (ADHD and anxiety, n = 551 in the diversity papers) and on EU-AIMS LEAP, where the Floris interaction replicated.
Metrics. We report AUC, sensitivity for autistic girls, single-attribute and multi-attribute directional amplification in both directions, and per-metric attribution. All come with bootstrap intervals.
Decisive check on the mechanism. A controlled injection test checks the audit itself. We add a site-specific variance shift to one metric in ABIDE data, correlated with diagnosis in training sites only. We then verify three things: accuracy-only selection picks that metric, the amplification audit flags it, and amplification-aware selection avoids it.
Decisive check on the claim. The core claim fails in either of two cases. It fails if accuracy-selected pipelines show no more amplification than random pipelines of similar accuracy. It also fails if variance residualization leaves amplification unchanged. In that event, whatever bias exists would come from somewhere other than proxy selection, and the proposal's diagnosis would be wrong.
References
- The Cost of Untracked Diversity in Brain-Imaging Prediction
- Population heterogeneity in clinical cohorts affects the predictive accuracy of brain imaging
- Towards robust and replicable sex differences in the intrinsic brain function of autism
- Directional Bias Amplification
- Men Also Do Laundry: Multi-Attribute Bias Amplification
- AutoML for NLP
- Why Fair Labels Can Yield Unfair Predictions: Graphical Conditions for Introduced Unfairness
- Disambiguating brain functional connectivity
- CellAgent: LLM-Driven Multi-Agent Framework for Natural Language-Based Single-Cell Analysis
- Spectral Graph Convolutions for Population-Based Disease Prediction
- Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
- FNNC: Achieving Fairness through Neural Networks