Who Gets Missed in the Tail? Thresholded Subgroup Underdiagnosis in Long-Tailed Chest X-ray Classification

Ha-Hieu Pham, Hai-Dang Nguyen, Dang P. M. Cao, Thanh-Huy Nguyen, Min Xu, Trung-Nghia Le, Ulas Bagci, Huy-Hieu PhamarXiv 2026 · arXiv

TL;DR

Standard evaluation metrics like macro-mAP often hide severe underdiagnosis of rare diseases within specific demographic subgroups when machine learning models are deployed in clinical workflows. This paper demonstrates that the score-to-decision thresholding step is a massive, underutilized lever for fairness: by implementing tail-aware thresholding, researchers significantly reduced missed positives for vulnerable subgroups without retraining the underlying model.

What Problem Does This Paper Address?

When deploying computer vision models for chest X-ray (CXR) triage, continuous risk scores must eventually be converted into binary decisions (e.g., "surface for immediate radiologist review" vs. "place in standard queue"). However, medical datasets are notoriously long-tailed; rare conditions have very few positive examples, and intersectional subgroups (e.g., a rare condition in a specific age or racial group) have fewer still.

The core gap the authors address is that standard performance metrics—like macro-mean Average Precision (macro-mAP)—evaluate the ranking of scores but completely mask what happens after a decision threshold is applied. A model can achieve an acceptable macro-mAP while simultaneously missing the vast majority of rare-positive cases within a vulnerable demographic subgroup. The authors formalize this pre-deployment fairness issue as thresholded class-subgroup underdiagnosis.

Methods

Rather than proposing a single new algorithm, the authors construct a diagnostic ladder consisting of six model variants (M1–M6) to isolate exactly where missed positives enter the clinical pipeline. All models share a standard ConvNeXt-Tiny backbone but vary in their loss functions, weighting schemes, or thresholding rules:

  • M1 (BCE): Standard Binary Cross-Entropy baseline.
  • M2 (ASL): Asymmetric Loss to mitigate class-level long-tail imbalances.
  • M3 (CB-ASL): Adds effective-number class weighting to ASL.
  • M4 (GroupDRO): A distributionally robust optimization baseline targeting group fairness.
  • M5 (GT-ASL): Group-Tail ASL, which multiplies ASL by a clipped class-subgroup weight based on training counts.
  • M6 (+TA): Reuses the exact same model scores as M5 but replaces standard $F_1$-optimal thresholds with Tail-Aware (TA) thresholding. For tail classes, it shifts the operating point to maximize worst-group recall, subject to a strict minimum precision floor ($\tau = 0.05$).

By comparing M5 and M6, the authors isolate the impact of the score-to-decision thresholding step from the model training process itself.

Datasets

The approach was audited across two major chest X-ray cohorts:

  1. VinDr-CXR: A dataset containing 15K training images and a 1,500-image test split with 15 labels (5 defined as tail classes). Subgroup metadata is limited to sex and age group extracted from DICOM headers.
  2. MIMIC-CXR/CXR-LT: A much larger dataset utilizing a 78,946-image test split with 40 labels (13 tail classes). This dataset links to Electronic Health Records (EHR), enabling richer subgroup analyses across sex, age, race, and insurance status.

Key Insights and Ideas

  • The Threshold is the Action Boundary: The transition from M5 to M6 proves that changing the decision rule for rare classes alters the False Negative Rate (FNR) far more effectively than complex, class-level long-tail training objectives.
  • Group Robustness $\neq$ Rare Subgroup Safety: Optimizing for aggregate group fairness (like GroupDRO) does not automatically protect rare subgroup cells. The audit showed that aggregate group robustness and thresholded rare-positive recall are entirely distinct dimensions.
  • Fairness is a Workflow Policy Choice: Lowering thresholds to catch more rare diseases inherently increases the "alert burden" (the number of cases flagged for review). The choice of where to set the precision floor ($\tau$) is a clinical workflow policy decision, not a static mathematical constant.

Key Results

The transition to tail-aware thresholding yielded dramatic reductions in missed cases across both datasets:

  • VinDr-CXR: Moving from the baseline (M1) to tail-aware thresholding (M6) reduced the overall tail FNR from 0.665 to 0.269. More importantly, it slashed the sex worst-group FNR from 0.705 to 0.157, and the age worst-group FNR from 0.822 to 0.133.
  • MIMIC-CXR/CXR-LT: The same tail-aware adjustment on M5 scores reduced the tail FNR from 0.866 to 0.741 and consistently lowered the worst-group FNR across all four demographic axes (sex, age, race, insurance).

The absolute reduction in missed positives per 100 true positives across these cohorts is visualized below:

Fig. 1. M5 to M6 threshold effect across cohorts. Grey markers show M5 with F1 thresholds; colored markers show the same score model after tail-aware thresholding. Numbers are absolute reductions in missed positives per 100 true positives.

However, this safety benefit comes at a clear operational cost. As shown in the operating-point trade-off curves, catching more missed positives requires flagging significantly more studies for human review:

Fig. 2. Operating-point trade-off for the M5 score model in both cohorts. Moving from the F1 threshold to tail-aware thresholds lowers missed tail positives but increases predicted tail-positive labels per 100 studies; the choice of τ is therefore a workflow policy rather than a universal setting.

Limitations and Future Work

The authors emphasize that their framework is a diagnostic measurement tool rather than a universal cure for algorithmic bias. Key limitations include:

  • Low Resolution in the Extreme Tail: In the VinDr-CXR test set, some tail classes contain as few as 3 to 19 positive cases, making disease-level conclusions statistically noisy.
  • Missing Metadata: Real-world clinical data is messy; for instance, DICOM headers in VinDr lacked informative sex data in 51% of cases and age data in 69% of cases.
  • Proxy Attributes: EHR-derived variables like race and insurance status function as imperfect administrative and social proxies rather than direct causal biological attributes.

Future work will need to explore how these thresholding policies perform under real-world clinical deployment constraints and across varying hospital architectures.

Summary updated

Abstract

In chest X-ray (CXR) classification, acceptable ranking performance can still leave rare-positive patients below threshold, especially within subgroups. We study this pre-deployment fairness problem as an audit question: after a long-tailed multi-label CXR model is converted from scores into decisions, who is missed? Across VinDr-CXR and MIMIC-CXR/CXR-LT, we use a diagnostic ladder to separate class-level long-tail losses, subgroup-aware weighting, group robustness, and threshold selection. On VinDr-CXR, group-tail weighting followed by tail-aware thresholding reduces tail FNR from 0.665 to 0.269, sex worst-group FNR from 0.705 to 0.157, and age worst-group FNR from 0.822 to 0.133, while macro-mAP increases from 0.611 to 0.635. On MIMIC-CXR/CXR-LT, the same score-to-threshold comparison reduces tail FNR from 0.866 to 0.741 and lowers worst-group FNR across sex, age, race, and insurance; residual missed-positive rates nevertheless remain high. Paired bootstrap contrasts on VinDr support the thresholded FNR reductions, and GroupDRO reference runs indicate that aggregate group robustness alone does not remove rare subgroup misses in this setting. The study supports a narrow audit claim: rare-label fairness in CXR depends jointly on the finding, subgroup, and operating threshold, not on label frequency or ranking metrics alone.
BibTeX
@misc{pham2026who,
  author = {Ha-Hieu Pham and Hai-Dang Nguyen and Dang P. M. Cao and Thanh-Huy Nguyen and Min Xu and Trung-Nghia Le and Ulas Bagci and Huy-Hieu Pham},
  title = {{Who Gets Missed in the Tail? Thresholded Subgroup Underdiagnosis in Long-Tailed Chest X-ray Classification}},
  year = {2026},
  eprint = {2607.07717},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2607.07717},
}
Figure