Abstract
Deep learning models for medical imaging require massive, diverse datasets to generalize effectively. However, integrating multiple public repositories often yields diminishing returns due to domain and label discrepancies. This paper introduces a novel framework that boosts chest X-ray (CXR) classification performance by effectively mining "imperfect" external data. By combining task-specific adversarial training (TAT) to align disparate image domains and uncertainty-aware temporal ensembling (UTE) to recover missing labels, the authors achieved a new state-of-the-art AUC of 0.8349 on the NIH ChestX-ray14 benchmark.
Introduction: The Challenge of Imperfect Data
While nearly a million CXR images are publicly available across repositories like NIH ChestX-ray14 and CheXpert, simply training on a combined pool of data is often ineffective. The authors identify two primary obstacles, termed "imperfect data":
- Domain Discrepancy: Images from different hospitals vary significantly in appearance due to hardware differences, imaging protocols (e.g., PA vs. Lateral views), and varied pre-processing techniques.
- Label Discrepancy: Datasets often use different "label spaces." For instance, NIH labels 14 diseases while CheXpert labels 13, with only 7 categories overlapping. This results in "partially labeled" images where a disease might be present but unrecorded because it was not the focus of that specific dataset.
To address these issues, the study utilizes three major datasets: NIH ChestX-ray14 (112,120 images), CheXpert (224,316 images), and ImsightCXR (a private dataset of 32,261 images used for validation).
Methodology: Mining Knowledge from Inconsistent Domains
The proposed framework employs a multi-pronged strategy to transform these imperfect datasets into a cohesive training signal. The core architecture utilizes DenseIBN-121, which integrates Instance Normalization to better handle style variations across multi-site medical data.
Task-specific Adversarial Training (TAT)
Instead of a single global discriminator to align datasets, the model uses a separate discriminator for each disease category shared between domains. This granular alignment forces the feature generator to produce domain-invariant features specifically relevant to each pathology, preventing the label distribution of one dataset from overwhelming the feature learning of another.
Uncertainty-aware Temporal Ensembling (UTE)
To handle missing labels, the model treats the unlabeled portions as a semi-supervised learning problem. It uses an Exponential Moving Average (EMA) of its own past predictions to generate "soft" pseudo-labels. To prevent error accumulation, an Uncertainty Filtering mechanism is implemented; pseudo-labels are only used when the model is confident, using a threshold that gradually decreases as the model stabilizes.
Results and Key Insights
The framework demonstrates significant improvements across multiple benchmarks:
- State-of-the-Art Performance: The model achieved an average AUC of 0.8349 on the NIH test set, surpassing previous benchmarks like CheXNet.
- Effective Knowledge Mining: When using NIH as the internal dataset and CheXpert as external data, the model showed significant gains in categories that were poorly represented in the original NIH set.
- Clinical Generalization: On the private ImsightCXR dataset, incorporating external NIH data boosted the average AUC from 0.884 to 0.901, proving that public datasets can significantly improve performance on local clinical data.
A key insight from this work is that "soft" labels are superior to "hard" labels (0 or 1) in the presence of domain shift, as they provide a smoother, more robust training signal. Furthermore, the curriculum-style learning enabled by uncertainty filtering allows the model to master certain cases before mining knowledge from more ambiguous images.
Conclusion
While the framework is robust, it faces limitations regarding label noise inherent in NLP-generated radiology reports and the computational scalability of maintaining multiple discriminators for TAT. Future work may incorporate patient metadata to further mitigate factors driving domain shift. Nonetheless, this research provides a powerful methodology for leveraging large-scale, imperfect public data to enhance medical diagnostic tools.
Summary updated