Can We Trust Deep Learning Based Diagnosis? The Impact of Domain Shift in Chest Radiograph Classification

Eduardo H. P. Pooch, Pedro L. Ballester, Rodrigo C. BarrosTIA @ MICCAI 2020 · OpenReview

Introduction

Deep learning models for medical imaging often boast "radiologist-level" performance in research settings. However, these models frequently struggle when deployed in real-world clinical environments. This discrepancy is largely driven by domain shift—the variation in data distribution between the training environment and the application site. This study evaluates four of the world’s largest chest X-ray datasets to quantify how these shifts impact diagnostic reliability.

The Problem: Beyond the i.i.d. Assumption

In standard machine learning, we assume that training and test data are "independent and identically distributed" (i.i.d.). In clinical practice, this assumption is rarely met. A chest X-ray from Stanford (CheXpert) differs from one taken in Spain (PadChest) due to variations in equipment settings, acquisition protocols, and patient demographics.

Pixel intensity distributions across the four major chest X-ray datasets, highlighting significant variations in data characteristics.

If a model is trained exclusively on one hospital's data, it may inadvertently learn the "signature" of that hospital’s equipment or specific labeling biases rather than the underlying pathology. The authors aim to measure the degradation in Area Under the Curve (AUC) when a model trained on one major public dataset is tested on another.

Methodology

The researchers employed an "all-vs-all" experimental design using the DenseNet121 architecture, replicating the CheXNet approach. Four separate models were trained—one for each dataset—and cross-evaluated against internal and external test sets.

Datasets and Harmonization

The study utilized over 800,000 images across four "hospital-scale" datasets:

  • ChestX-ray14 (NIH): 112,120 frontal-view images.
  • CheXpert (Stanford): 224,316 images using an automated NLP labeler.
  • MIMIC-CXR (MIT): 371,920 images sharing the same NLP labeler as CheXpert.
  • PadChest (Spain): 160,000 images with labels extracted from Spanish reports and unique visual artifacts.

To ensure comparability, the authors mapped findings to a common set of eight radiological labels and standardized image preprocessing (224x224 resizing and ImageNet normalization).

Key Insights: The "Home Field" Advantage

Models consistently achieved their highest performance on their own test sets, suggesting a degree of "overfitting" to institutional characteristics.

Average image representations for each dataset (I: ChestX-ray14, II: CheXpert, III: MIMIC-CXR, IV: PadChest), illustrating distinct institutional visual signatures.

Asymmetric Generalization

Generalization capabilities varied significantly between datasets. Models trained on CheXpert and MIMIC-CXR generalized better than those trained on ChestX-ray14. This is likely due to the "Labeler Effect": CheXpert and MIMIC-CXR use the same automated NLP tool for ground truth extraction, creating a consistent logic that aids cross-dataset performance.

The Reliability Gap

The most dramatic performance drop occurred with the ChestX-ray14 model, which saw its mean AUC plummet from 0.83 (internal) to 0.68 (external on CheXpert). Such a significant decline represents the difference between a clinically useful tool and one that is potentially misleading.

Conclusion and Future Directions

The study highlights that high AUC scores in research papers do not guarantee safety in local clinics. Automated NLP labelers, while efficient, introduce a 10-30% error rate that limits model performance. Furthermore, the reliance on frontal views ignores critical diagnostic information found in lateral X-rays.

The authors suggest that the industry should move away from "one-size-fits-all" models. Instead, the focus should shift toward fine-tuning: pre-training models on massive datasets like MIMIC-CXR and calibrating them with high-quality, local data from the deployment site. External validation is a necessity, not an option, for building trustworthy AI in medicine.

Summary updated

Abstract

While deep learning models become more widespread, their ability to handle unseen data and generalize for any scenario is yet to be challenged. In medical imaging, there is a high heterogeneity of distributions among images based on the equipment that generates them and their parametrization. This heterogeneity triggers a common issue in machine learning called domain shift, which represents the difference between the training data distribution and the distribution of where a model is employed. A high domain shift often results in a poor generalization performance from the models. In this work, we evaluate the extent of which domain shift damages model performance on four of the largest datasets of chest radiographs. We show how training and testing with different datasets (e.g., training in ChestX-ray14 and testing in CheXpert) drastically affects model performance, posing a big question over the reliability of deep learning models trained on public datasets. We also show that models trained on CheXpert and MIMIC-CXR generalized better to other datasets.
BibTeX
@inproceedings{pooch2020can,
  author = {Eduardo H. P. Pooch and Pedro L. Ballester and Rodrigo C. Barros},
  title = {{Can We Trust Deep Learning Based Diagnosis? The Impact of Domain Shift in Chest Radiograph Classification}},
  booktitle = {TIA @ MICCAI},
  year = {2020},
  url = {https://openreview.net/forum?id=mwBI4c2h-iW},
}
Figure