Medical image classifiers often score well by reading signs of care rather than signs of disease. A chest-tube detector looks like a pneumothorax detector. A model that spots surgical skin markings looks like a melanoma detector. The trouble goes beyond falling accuracy on cases without these cues. A chest tube means a clinician has already diagnosed and treated the patient. So when such a model is credited with "improving risk stratification," much of that credit is borrowed from knowledge the clinic already had.
Clinical statistics has a standard tool for this kind of question. The net reclassification index (NRI) counts how often adding a new predictor to an existing risk model moves patients who had the event toward higher risk and patients who did not toward lower risk. We propose applying it with one specific change. The shortcut itself (tube present, ruler present, pediatric age, acquisition site) goes into the baseline model. The image model then gets credit only for reclassification it achieves beyond the shortcut. Gradient saliency maps, which show which pixels drive a prediction, are used to find candidate shortcuts when nobody has labeled them.
Core Claim
The claim has two parts.
The first part concerns measurement. "Shortcut-adjusted" NRI and its continuous companion, the integrated discrimination improvement (IDI), are computed on an ordinary, shortcut-contaminated test set. We claim this adjusted NRI ranks candidate models by their true robustness about as well as the shortcut-free test sets that are usually unavailable, and better than aggregate AUROC or unadjusted NRI. AUROC is the area under the ROC curve, a measure of how well a model ranks positives above negatives.
The second part concerns discovery. Shortcut indicators found from saliency maps, with only minutes of human naming, recover most of the gap that true shortcut labels reveal.
To make this precise, let $M$ be the available metadata (age, sex, view position), $S$ the shortcut indicator(s), and $f(x)$ the image model's score. Define
$$\text{NRI}(f \mid S, M) = \text{NRI}\big(\, p(Y \mid S, M, f(x)) \ \text{vs.}\ p(Y \mid S, M) \,\big),$$
and the borrowed reclassification
$$B(f) = \text{NRI}(f \mid M) - \text{NRI}(f \mid S, M).$$
$B(f)$ is the part of the image model's apparent reclassification that the shortcut alone would have supplied. If a model only reads the shortcut, its $B(f)$ should be close to its whole NRI and its adjusted NRI close to zero. A model that reads pathology should keep a positive adjusted NRI.
Why This, and What Is Already Done
Three bodies of work frame the problem.
Two papers motivate it by showing that aggregate metrics hide shortcut failure.
- Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging showed that pneumothorax AUROC on ChestX-ray14 was 0.94 with drains present and 0.77 without.
- Critical evaluation of deep neural networks for wrist fracture detection found AUROC fell from 0.99 to 0.84 on CT-confirmed hard cases. Its deep ensembles did not flag those cases.
Neither paper asks what a model contributes beyond information the clinic already holds.
Two papers supply training methods whose outputs we will evaluate.
- Reducing Reliance on Spurious Features in Medical Image Classification with Spatial Specificity supervises training with boxes or masks. It raised robust AUROC on SIIM-ACR pneumothorax from 0.46 to 0.82.
- Unlearning Spurious Correlations in Chest X-ray Classification applies a triplet loss to Grad-CAM maps. Grad-CAM is a coarse saliency map built from last-layer convolutional features. This method traded accuracy (0.95 to 0.90) for slightly better attention inside the lungs (activation precision 0.32 to 0.35). The paper could not say whether the lost five points were real errors or correct removal of shortcut credit.
Both papers judge success by discrimination alone. Neither connects to reclassification.
Two further papers constrain how saliency can be used.
- Benchmarking saliency methods for chest X-ray interpretation found saliency maps localize pathology 24.0% worse than radiologists in mIoU, and 76.2% worse for lung lesions.
- Robust Attribution Regularization. showed that attributions are fragile.
Because of these results, we never use saliency maps as weights inside the metric, and we never use them as evidence of correctness. We use them only to propose candidate shortcut variables. Those candidates are then tested statistically by asking whether adding them to the baseline absorbs the image model's reclassification.
The file shows the obvious combination, "train with a saliency loss, report NRI," has not been done. It is also shallow, because it simply bolts a metric onto an existing method. Our proposal differs in two ways. The shortcut moves into the reference model, which is the standard incremental-value logic of the tabular biomarker literature, for example Risk prediction of early-onset myocardial infarction using plasma proteomics, conventional risk factors, and polygenic risk score. And the whole construction is validated against shortcut-free ground truth.
Research Program
Building the measurement. We fit $p(Y \mid S, M)$ and $p(Y \mid S, M, f(x))$ as logistic models on a held-out calibration split. The image score enters as its logit. $S$ enters either as a binary label or as the probability from a small auxiliary classifier trained to detect the artifact. The probability form lets the baseline capture finer artifact information, such as tube type or position, that a deep model might otherwise exploit as residual leakage.
The literature lists two known weaknesses of NRI, and we address both directly:
- Threshold sensitivity of category NRI. There are no agreed pneumothorax or melanoma risk strata. We therefore report category NRI as a curve over a pre-registered range of cutoffs rather than at a single cutoff.
- Inflation of continuous NRI by tiny probability shifts. We use a margin version that counts only shifts larger than $\delta$, reported for several values of $\delta$.
We also report the event and non-event components of NRI separately. This split is what can explain the eXBL accuracy drop. Shortcut removal should show up as downgrades among treated positives, whose credit was borrowed, and upgrades among untreated positives.
Discovering shortcuts from saliency. For each training image we compute Grad-CAM, which Saporta et al. found best on CheXlocalize, and Integrated Gradients. We then delete the attribution that falls inside an anatomy mask:
- lung fields, from a generic lung segmenter, for chest X-rays;
- the lesion, from ISIC's lesion masks, for dermoscopy.
We cluster the image patches with the highest remaining out-of-anatomy attribution in a pretrained embedding space. A reviewer looks at about twenty clusters and names any that are artifacts. This takes minutes, compared with the dense masking Saab et al. needed. Each named cluster becomes a patch detector, and the detector's output serves as a candidate $S$.
Bridging Explainability and Embeddings: BEE Aware of Spuriousness pursues discovery without counterexamples in weight space. We do it in attribution space and add a statistical check that BEE lacks: a candidate counts as a real shortcut only if adding it to the baseline significantly lowers the image model's adjusted NRI.
Building a model pool with known robustness. On each dataset we train a pool of models whose shortcut reliance varies in a controlled way:
- plain training on binary labels;
- Saab-style supervision with 0, 600, and full masks;
- eXBL Grad-CAM refinement;
- a lightweight Integrated-Gradients penalty on out-of-anatomy attribution, with the expensive second-order robust version applied to a subset only;
- models trained on resampled data with the shortcut–label correlation set between 0.5 and 0.95.
Each model's true robustness is its AUROC on a shortcut-free test subset.
Baselines
Naive reference: aggregate AUROC and unadjusted NRI. The first baseline ranks the model pool by aggregate AUROC on the ordinary test set, as almost all imaging papers do. The second ranks by NRI over a metadata-only baseline, as the tabular literature does. The cited results predict both will reward shortcut users. Oakden-Rayner et al. report aggregate pneumothorax AUROC near 0.94 even though tube-absent AUROC is 0.77. Both baselines use exactly the same test images and calibration split as our metric.
Strongest existing ranking signal: robust AUROC on a shortcut-free subset. This follows Saab et al., who report robust AUROC of 0.82 with full segmentation supervision on SIIM-ACR pneumothorax and 0.73 on ISIC melanoma. Building the shortcut-free subset requires artifact labels, so it is the gold standard rather than a free competitor.
Our metric beats the practical alternatives if two conditions hold, both measured across the model pool as rank correlation with robust AUROC:
- its rank correlation with robust AUROC is significantly higher than that of aggregate AUROC and of unadjusted NRI;
- it comes close to the correlation that within-stratum AUROC attains.
Within-stratum AUROC is the competitor that uses the same labels we use (see below). Matching it on ranking while adding a decision-level decomposition is the minimum bar.
Ablations. Four ablations isolate the key ingredients:
- What goes in the baseline: true $S$ labels; saliency-discovered $S$; and an equal number of clusters drawn at random from high-attribution patches, as a placebo.
- Which saliency method does the discovery: Grad-CAM versus Integrated Gradients.
- How $S$ is encoded: binary label versus learned artifact probability.
- Continuous NRI margin: with versus without $\delta$, to show whether inflation changes the rankings.
All variants are fit with the same tuning budget on the same splits.
Why not just…?
Why not just report AUROC separately within shortcut-present and shortcut-absent subgroups? This is the most serious threat, and we treat it as a co-equal competitor rather than a straw man. When true $S$ labels exist, stratified AUROC may rank models just as well. Our case rests on two points.
- NRI's event and non-event components say which patients a debiasing step moves, and whether they cross decision cutoffs. That is exactly what the eXBL accuracy drop leaves unexplained, and AUROC is a rank measure blind to such shifts.
- When subgroup labels are missing, subgroup AUROC is impossible. Oakden-Rayner et al. note that completing the label schema is often infeasible.
If within-stratum AUROC ranks models as well as adjusted NRI, and the component analysis tells clinicians nothing that subgroup sensitivity does not, we will report that the measurement contribution reduces to subgroup analysis. The discovery contribution would still be needed to supply the strata.
Why not just annotate the artifacts, or train with masks as Saab et al. did? Masks address training, not evaluation. Even a mask-trained model needs an evaluation that separates borrowed credit from earned credit. The labeling cost is also real. The question this proposal can settle is how much of the true-$S$ result survives when $S$ comes from saliency clusters named in minutes. The placebo-cluster ablation shows whether that recovery is genuine or just extra covariates soaking up variance.
Why not just look at the saliency maps, or score their overlap with anatomy, as Singh et al. did? Agreement Between Saliency Maps and Human-Labeled Regions of Interest: Applications to Skin Disease Classification proposed overlap as a gatekeeper. Saporta et al. found maps localize far worse than radiologists, and Chen et al. found attributions can be shifted without changing predictions. We include each model's mean out-of-anatomy attribution fraction as an additional ranking signal in the main experiment. If it predicts robust AUROC as well as adjusted NRI does, the reclassification step is unnecessary.
Why not just flag low-confidence cases? Raisuddin et al. found that deep ensembles gave confident wrong answers on the hard wrist cases. For that reason we expect ensemble disagreement to rank poorly. We include it in the comparison so the claim is tested rather than assumed.
Proposed Experiments and Decisive Checks
Datasets. We use four datasets:
- SIIM-ACR pneumothorax, with the chest-tube labels used by Saab et al.;
- NIH ChestX-ray14 pneumothorax, with drain labels in the style of Oakden-Rayner et al.;
- ISIC melanoma, with surgical markings and rulers;
- the multi-source COVID-19 chest X-ray set, where pediatric age is the confound.
Activation precision on CheXlocalize, the radiologist-segmented chest X-ray benchmark, serves as a secondary check on attribution quality. Wrist and hip fracture cohorts are out of scope because we lack access to them.
First decisive check. Across each model pool, shortcut-adjusted NRI and IDI computed on the ordinary test set must correlate with shortcut-free robust AUROC significantly more strongly than aggregate AUROC, unadjusted NRI, saliency overlap, and ensemble disagreement. Significance is assessed by bootstrap over models and patients.
Second decisive check. Saliency-discovered $S$ must recover a large share of the borrowed reclassification $B(f)$ measured with true $S$. Placebo clusters must recover essentially none. For the pediatric confound, the epiphysis cluster should be found without being named in advance.
Third check. On eXBL and on mask-supervised models, the event and non-event components should show the predicted signature: treated positives move down, untreated positives move up. Showing this pattern would explain the reported 0.95 to 0.90 drop instead of merely tolerating it. The dermatology confounds are diffuse anatomy (lips, hair) rather than discrete objects, so they are where discovery is most likely to fail. A failure there would mark the boundary of the method, and we will report it as such.
References
- Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging
- Critical evaluation of deep neural networks for wrist fracture detection
- Reducing Reliance on Spurious Features in Medical Image Classification with Spatial Specificity
- Unlearning Spurious Correlations in Chest X-Ray Classification
- Benchmarking saliency methods for chest X-ray interpretation
- Robust Attribution Regularization.
- Risk prediction of early-onset myocardial infarction using plasma proteomics, conventional risk factors, and polygenic risk score
- Bridging Explainability and Embeddings: BEE Aware of Spuriousness
- Agreement Between Saliency Maps and Human-Labeled Regions of Interest: Applications to Skin Disease Classification