Rescuing referral failures during automated diagnosis of domain-shifted medical images

Anuj Srivastava, Karm Patel, Pradeep Shenoy, Devarajan SridharanarXiv 2023 · DOI · arXiv

Introduction: The Challenge of Selective Classification

In a clinical environment, the ideal automated diagnostic tool functions as a collaborative partner: it handles straightforward cases with high confidence and refers "uncertain" or complex cases to a human expert. This paradigm, known as selective classification, is essential for maintaining safety and trust in AI-driven healthcare. However, deep learning models are notoriously sensitive to covariate shift—changes in the input data distribution that occur when a model trained on one population (e.g., high-resolution scans from a urban US hospital) is deployed on another (e.g., lower-quality images from a rural clinic in India).

Conceptual framework of selective classification and the taxonomy of methods used to address referral failures.

The Paradox of Referral Failure

The authors of this study identified a startling phenomenon they term referral failure. Under normal conditions, as the referral rate increases (i.e., as more cases are sent to a doctor), the model’s accuracy on the remaining, non-referred cases should monotonically increase. However, under domain shift, this relationship breaks down. The authors observed non-monotonic referral curves, where accuracy actually drops as more cases are referred.

This failure occurs because the model becomes "confidently wrong" about certain out-of-distribution (OOD) images. It retains these incorrect but high-confidence predictions in its "certain" pool while referring away correctly predicted cases that happen to have lower confidence scores. This effectively tanks the performance of the human-AI system precisely when the model encounters unfamiliar data.

Methodological Innovations

To combat these failures, the researchers evaluated several state-of-the-art (SOTA) approaches and proposed two primary strategies to "rescue" the referral process:

  1. REMEDIS/DAN (Domain Adversarial Network): By integrating a Domain Adversarial Network into the REMEDIS framework (which utilizes self-supervised pretraining), the authors forced the model to learn features that are useful for diagnosis but invariant to the source domain. The DAN acts as a discriminator that tries to identify the image's origin; the diagnostic model is trained to "fool" this discriminator, ensuring it doesn't rely on domain-specific "style" or hardware artifacts.
  2. ViT+SR (Split Referral): This is a novel post-hoc strategy designed to address uncertainty imbalance. Often, one class (e.g., "Healthy") may naturally exhibit higher entropy or lower confidence than another (e.g., "Diseased") in OOD data. Standard referral uses a single threshold, which might cause the model to disproportionately refer away one class. Split Referral ranks samples separately for each predicted class, ensuring a balanced referral process that preserves the overall performance metric.

Experimental Results and Key Insights

The study utilized two high-stakes medical imaging tasks: Diabetic Retinopathy (DR) using the EyePACS and APTOS datasets, and Chest X-ray (CX) using CheXpert and ChestX-ray14.

Empirical results showing non-monotonic referral curves and the performance gains achieved by REMEDIS and MCD.

As shown in the results, the proposed methods significantly outperformed standard baselines. REMEDIS/DAN and ViT+SR achieved improvements of typically >10% in the Area Under the Referral Curve (AURC). In the Diabetic Retinopathy task, REMEDIS/DAN achieved an AURC-AUROC of 0.98, a 12.8% gain over the ViT-Base. In Chest X-ray tasks, both methods successfully rescued the non-monotonic curves, restoring the expected behavior where higher referral rates lead to higher system accuracy.

One critical insight was the "Contrastive Learning Trap." The authors found that popular self-supervised methods like SimCLR can actually exacerbate referral failure by increasing the segregation between in-domain and out-of-domain data in the model's internal representations. This highlights the need for domain-invariant learning rather than just general-purpose self-supervision.

Conclusion

Rescuing referral failures is a prerequisite for the safe deployment of AI in global health. By focusing on learning domain-invariant features (REMEDIS/DAN) and correcting for uncertainty imbalances (Split Referral), this research provides a robust roadmap for building diagnostic tools that truly "know what they don't know," even when faced with the unpredictable shifts of real-world clinical data.

Summary updated

Abstract

The success of deep learning models deployed in the real world depends critically on their ability to generalize well across diverse data domains. Here, we address a fundamental challenge with selective classification during automated diagnosis with domain-shifted medical images. In this scenario, models must learn to avoid making predictions when label confidence is low, especially when tested with samples far removed from the training set (covariate shift). Such uncertain cases are typically referred to the clinician for further analysis and evaluation. Yet, we show that even state-of-the-art domain generalization approaches fail severely during referral when tested on medical images acquired from a different demographic or using a different technology. We examine two benchmark diagnostic medical imaging datasets exhibiting strong covariate shifts: i) diabetic retinopathy prediction with retinal fundus images and ii) multilabel disease prediction with chest X-ray images. We show that predictive uncertainty estimates do not generalize well under covariate shifts leading to non-monotonic referral curves, and severe drops in performance (up to 50%) at high referral rates (>70%). We evaluate novel combinations of robust generalization and post hoc referral approaches, that rescue these failures and achieve significant performance improvements, typically >10%, over baseline methods. Our study identifies a critical challenge with referral in domain-shifted medical images and finds key applications in reliable, automated disease diagnosis.
BibTeX
@misc{srivastava2023rescuing,
  author = {Anuj Srivastava and Karm Patel and Pradeep Shenoy and Devarajan Sridharan},
  title = {{Rescuing referral failures during automated diagnosis of domain-shifted medical images}},
  year = {2023},
  eprint = {2311.16766},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.16766},
}
Figure