Medical imaging models are usually checked for fairness using a handful of broad categories such as "White," "Black," and "Asian." They are also usually checked at the hospital where they were built. Both habits hide failures. In emergency-department risk scores, performance differences inside a broad race category, for example between Korean and South Asian patients, often exceed the differences between categories (Coarse race data conceals disparities in clinical risk score performance). Separately, a chest X-ray classifier's average score can collapse when it moves from one hospital to another (Can We Trust Deep Learning Based Diagnosis?). Nobody has asked whether these two failures compound. A model trained at Stanford could look fair across broad race groups at a Boston hospital while quietly failing, say, Brazilian or Southeast Asian patients whose images and disease patterns the shift affects most.
We propose to measure this directly and then fix the part of the pipeline where the evidence says fixes actually work, which is model selection. There is a natural test bed. MIMIC-CXR comes from the same Boston hospital as MIMIC-IV, whose self-reported race field records the fine-grained groups used in the emergency-department study. CheXpert, from Stanford, records only broad race. We would therefore train at Stanford, deploy in Boston, and score every model against fine-grained labels that neither training nor model selection ever sees. That setup lets us test a practical claim. Most deployment sites lack fine-grained labels, but a selection rule that hunts for poorly served clusters inside each broad group of the target site's image embeddings may recover most of what fine-grained labels would reveal.
Core Claim
We make two claims. The first is an empirical measurement. We define the masking gap as the drop from worst coarse-group AUROC (area under the ROC curve) to worst granular-group AUROC, restricted to granular groups above a minimum patient count. We claim this gap is larger after cross-site transfer than within a single site, by more than chance variation in small groups can explain. The second claim is methodological. Model selection is where fairness is won or lost: MEDFAIR found that the selection criterion mattered more for worst-group performance than the choice of debiasing algorithm. We therefore claim that a selection criterion built from label-free, coarse-group-conditioned embedding clusters picks models with better true granular worst-group AUROC than selection by overall AUROC or by coarse worst-group AUROC.
The second claim rests on a condition we test rather than assume. Chest X-rays reveal broad race, sex, and age with AUROC above 0.90 (The Role of Subgroup Separability). That paper also showed that when a model cannot tell groups apart, harm spreads across everyone and disparity metrics look deceptively fair. If fine-grained ethnicity is not recoverable from the image embeddings within a broad group, embedding clusters cannot find those patients, and the proxy will fail. We report this outcome either way.
Relation to Prior Work
Two papers motivate the gap. Coarse race data conceals disparities shows granular masking in tabular emergency-department data at one site, but it neither studies images nor proposes a mitigation. Can We Trust Deep Learning Based Diagnosis? quantifies cross-site collapse but tracks no demographics at all. Our work is the imaging, cross-site analogue of the first paper, scored at the granularity the second paper ignores.
Four fairness papers constrain the design, and all four stop at coarse attributes. MEDFAIR benchmarks eleven methods, including a CheXpert-to-MIMIC transfer. Its central findings are that debiasing methods rarely beat plain training, that weight averaging over flat regions of the loss (SWAD) ranks best, and that selection criteria dominate. We reuse its transfer direction and its focus on selection, but we score at granular level. Improving the Fairness of Chest X-ray Classifiers shows that simple group-balanced training matches worst-case optimization in-distribution. It also shows that automated labeler errors track patient age, a confound we must control. How Fair are Medical Imaging Foundation Models? shows that medical pretraining raises accuracy but widens broad racial gaps, so we include a medical-pretrained backbone and an ImageNet-pretrained backbone as separate candidates.
Fairness and Accuracy under Domain Generalization proves that fairness transfer requires the representation to be independent of site given both label and attribute. It then enforces this with image-translation networks, using binary sex or binned age and synthetic shifts. We deliberately do not build a conditional-matching method over 20-plus granular groups. The per-cell sample sizes make such density estimation implausible, and the fine-grained labels are absent at the source anyway. We use that paper's theory only to predict where masking should appear: in granular groups whose image-given-label distribution differs across sites.
Deep Hypothesis Tests Detect Clinically Relevant Subgroup Shifts detects population-level shifts from 100 to 200 unlabeled images using a classifier that tries to tell source from target (a two-sample test). We borrow its idea of looking for structure without labels, but we look inside each coarse group and use what we find to rank models, not to raise an alarm.
Research Program
We would build a pool of candidate models trained on CheXpert. The pool covers two backbones (ImageNet-pretrained and chest-X-ray-pretrained), four training methods (plain training, training rebalanced by coarse race, GroupDRO over coarse race, and SWAD), and several seeds and learning rates, giving roughly 60 to 100 checkpoints. GroupDRO is a method that minimizes the loss of the worst-off group. The tasks are "No Finding," pleural effusion, and cardiomegaly. We choose these because the MIMIC-CXR paper reports high labeler agreement on some of them: F1 of 0.973 for pleural effusion (MIMIC-CXR). High agreement limits label noise as an explanation for gaps.
MIMIC-CXR is split by patient into a target selection set and a target test set. Both carry coarse race; fine-grained race is attached only for scoring. As a within-site reference, we train and test the same pool entirely inside MIMIC-CXR.
The proposed selection criterion works as follows. For each candidate model, we take the model's own penultimate-layer embeddings of the target selection images. Within each coarse race group, we cluster them into $K$ clusters, choosing $K$ so that cluster sizes roughly match the granular group sizes that are plausible at a site; we sweep $K$ as a sensitivity analysis. The criterion is the lowest cluster-level AUROC, shrunk toward the coarse-group AUROC in proportion to its bootstrap variance, so that tiny noisy clusters cannot dominate.
The analysis has three parts, all using patient-level bootstrap intervals. The first estimates the masking gap within-site and cross-site. We compare it against a permutation null that shuffles granular labels within each coarse group, which separates real heterogeneity from small-sample noise. The second is a probe of granular separability: we fit a linear classifier on frozen embeddings to predict granular group within each coarse group. This tells us whether the precondition for the clustering proxy holds. The third evaluates selection rules by the true granular worst-group AUROC of the model each rule picks on the held-out test set, and by rank correlation with that quantity across the pool.
Baselines
The naive reference is selection by overall target AUROC with no demographic information. This is what most deployments do, and it tells us how much granular harm goes unnoticed by default. The within-site reference, where models are trained and tested inside MIMIC-CXR, isolates how much of the masking gap is caused by the shift rather than already present at home. For scale, Can We Trust Deep Learning Based Diagnosis? reports mean AUROC falling from 0.83 internally to 0.68 on CheXpert for a ChestX-ray14-trained DenseNet121.
The strongest existing competitor is MEDFAIR-style minimax selection by coarse worst-group AUROC, applied to the same pool, which includes SWAD. MEDFAIR reports that switching from overall-performance selection to minimax Pareto selection improved the average rank of worst-group performance from 2.53 to 1.49 (MEDFAIR). This is the right competitor because it is the best documented lever for worst-group performance under this exact transfer. It also uses exactly the information our criterion uses (the target selection set with coarse race), so the comparison is fair by construction: same checkpoints, same selection images, same compute. We found no reported result on Lacuna for granular worst-group AUROC of any chest X-ray model, so the bar for that metric is the one this study establishes. Beating the competitor means a statistically reliable increase, by paired bootstrap over test patients, in the true granular worst-group AUROC of the selected model, with no more than a small loss in overall AUROC.
Two ablations isolate the key ingredient, which is conditioning the clusters on coarse group. The first clusters embeddings without first splitting by coarse race. The second replaces clusters with random partitions of the same sizes, which shows how much a plain small-group variance penalty achieves on its own. An oracle that selects using true granular labels on the selection set gives the ceiling.
Why not just…?
Why not just use coarse worst-group selection, since granular gaps may be small-sample noise? This is the most serious threat. The permutation null is designed to settle it. If the observed within-coarse spread of granular AUROC is not larger than the spread under shuffled granular labels, the masking claim fails for imaging. We would then report coarse selection as sufficient, a useful negative result.
Why not just rebalance training or run GroupDRO over coarse race? Both are already in the candidate pool. In-distribution, Improving the Fairness of Chest X-ray Classifiers found rebalancing competitive with worst-case methods. But upweighting a coarse group mostly upweights its largest subgroup, as the emergency-department study's variance findings imply. The pool comparison shows directly whether coarse-trained methods shrink the granular gap after transfer.
Why not just infer granular ethnicity and audit with it? Name-based inference (Enabling disaggregation of Asian American subgroups) needs names, which de-identified imaging archives do not have. Inferring ethnicity from pixels is exactly what the separability probe measures. If that probe is strong, the clustering proxy should also work. If it is weak, both fail, and the honest conclusion is that granular labels must be collected at the site.
Why not just recalibrate outputs after deployment? Avoiding Catastrophic Referral Failures shows that site-level recalibration can repair referral behavior. However, recalibration applies one transform to all patients and cannot restore ranking quality lost for a subgroup. AUROC measures ranking and does not change under any monotone rescaling of scores, so it is exactly the metric recalibration cannot improve. We report calibration error per granular group as a secondary check.
Decisive Checks
Four results decide the study. The first is whether the cross-site masking gap exceeds both the within-site gap and the permutation null. We expect this to appear most strongly for granular groups that differ from their coarse group in the proportion of portable anteroposterior films, the X-ray view taken at the bedside. We therefore stratify every result by view position and report the group-by-view breakdown, to avoid mistaking an acquisition confound for an ethnicity effect.
The second is whether the within-coarse separability probe is clearly above chance. The third is whether the coarse-conditioned cluster criterion beats coarse minimax selection on true granular worst-group AUROC, and beats both ablations. The fourth addresses labeler bias. We repeat the analysis on pleural effusion, where the labeler is most reliable. If the gap survives there but not on "No Finding," labeler bias rather than the image model is the likely cause, and we would say so.
References
- Coarse race data conceals disparities in clinical risk score performance
- Can We Trust Deep Learning Based Diagnosis? The Impact of Domain Shift in Chest Radiograph Classification
- MEDFAIR: Benchmarking Fairness for Medical Imaging
- The Role of Subgroup Separability in Group-Fair Medical Image Classification
- Improving the Fairness of Chest X-ray Classifiers
- How Fair are Medical Imaging Foundation Models?
- Fairness and Accuracy under Domain Generalization
- Deep Hypothesis Tests Detect Clinically Relevant Subgroup Shifts in Medical Images
- MIMIC-CXR: A large publicly available database of labeled chest radiographs
- Enabling disaggregation of Asian American subgroups: a dataset of Wikidata names for disparity estimation
- Avoiding Catastrophic Referral Failures In Medical Images Under Domain Shift