Discovery of a Generalization Gap of Convolutional Neural Networks on COVID-19 X-Rays Classification

Kaoutar Ben Ahmed, Gregory M. Goldgof, Rahul Paul, Dmitry B. Goldgof, Lawrence O. HallIEEE Access 2021 · DOI

TL;DR

This paper reveals a significant generalization gap in deep learning models for COVID-19 detection from X-rays. Despite achieving near-perfect accuracy on data from the same sources used for training, these models perform poorly on unseen data, suggesting they learn "shortcuts" (confounding features) related to image acquisition rather than true COVID-19 indicators. This highlights the critical need for robust external validation before clinical deployment.

What Problem Does This Paper Address?

The COVID-19 pandemic spurred rapid development of AI models, particularly deep learning (DL) approaches, for diagnosing the disease using chest X-rays (CXRs). Many studies reported exceptionally high accuracy, leading to optimism about AI's potential in clinical settings. However, a crucial question remained: do these models truly learn to identify COVID-19 pathology, or are they exploiting superficial, data-source-specific features (confounders) that don't generalize to new datasets or clinical environments? This paper directly addresses this by investigating the generalization capabilities of DL models trained on CXRs, aiming to determine if they learn medically relevant features or rely on spurious correlations.

Methods

The researchers employed a fine-tuned ResNet50 convolutional neural network (CNN) as their primary model. They experimented with two fine-tuning strategies: one where only the final layers were trainable (Finetune1) and another where the last two convolutional layers were also unfrozen (Finetune2). They also tested a VGG-16 model for comparison. A key methodological step involved pre-processing the CXR images by segmenting and cropping the lung area to reduce noise and potential artifacts from image borders, aiming to focus the model on relevant anatomical regions. Data augmentation techniques, including rotations and horizontal flipping, were used to increase the training dataset size.

Datasets

The study utilized several publicly available datasets for both COVID-19 positive and pneumonia (non-COVID-19) cases.

  • COVID-19 Datasets:
    • BIMCV-COVID-19+ (Spain)
    • COVID-19-AR (USA)
    • V2-COV19-NII (Germany)
  • Pneumonia Datasets:
    • National Institute of Health (NIH) dataset
    • Chexpert dataset
    • Padchest dataset

A critical aspect of their experimental design was the careful separation of "seen" (training/validation) and "unseen" (testing) data sources to rigorously evaluate generalization. They specifically avoided datasets with significant age discrepancies (e.g., pediatric pneumonia vs. adult COVID-19) to minimize age-related confounding features.

Key Insights and Ideas

  • The "Generalization Gap" is Real and Significant: The most striking finding is the dramatic drop in performance when models are tested on data from sources not seen during training. This suggests that high accuracy on a single dataset is not indicative of real-world utility.
  • Confounding Features are a Major Problem: The models appear to be learning features related to the origin of the X-ray (e.g., specific machine types, imaging protocols, or even subtle differences in image processing) rather than the underlying pathology of COVID-19. This is akin to a radiologist identifying a patient's location based on the hospital's letterhead rather than their medical chart.
  • Raw Data and Careful Pre-processing are Crucial: While the authors attempted to mitigate some confounding factors through lung cropping and using raw data where possible, the results indicate that subtle differences in data acquisition and processing can still lead models astray.
  • Fine-tuning Can Improve Local Performance: Adapting a model to a specific, unseen data source through fine-tuning can significantly boost performance on that local data, suggesting that models can be tailored for specific clinical environments, but this doesn't guarantee broader generalization.

Key Results

  • Models achieved near-perfect Area Under the Curve (AUC) scores (e.g., 1.00 and 0.96) when tested on data from the same sources used for training.
  • When tested on completely unseen data sources, the AUC dropped dramatically, reaching as low as 0.38 (worse than random guessing) in some cases.
  • A statistically significant difference (p-value < 1.78 x 10^-70) was observed between performance on seen and unseen data sources, confirming the generalization gap.
  • Fine-tuning a model on a small subset of unseen data improved its performance on that specific unseen source, but the extent to which it learned medically relevant features remains unclear.

Limitations and Future Work

The authors acknowledge several limitations:

  • Limited Data Sources: While they used multiple sources, they couldn't cover all published papers, and the availability of truly diverse and well-annotated external datasets is limited.
  • Unknown Pre-processing: For some datasets, the exact pre-processing steps were unknown, making it difficult to ensure uniform normalization.
  • Inconclusive Grad-CAM Results: Visualizations using Grad-CAM to highlight model focus areas were inconclusive after fine-tuning, making it hard to definitively confirm if models were learning true pathological features or still relying on confounders.

Future work should focus on:

  • Developing standardized benchmarks with strictly separated training and testing data sources to better evaluate generalization.
  • Investigating features that generalize across sources to ensure models are learning medically relevant markers.
  • Exploring methods to explicitly mitigate confounding features during model training.
  • Combining CXR analysis with other clinical data to build more robust diagnostic systems.

Summary updated

BibTeX
@article{ahmed2021discovery,
  author = {Kaoutar Ben Ahmed and Gregory M. Goldgof and Rahul Paul and Dmitry B. Goldgof and Lawrence O. Hall},
  title = {{Discovery of a Generalization Gap of Convolutional Neural Networks on COVID-19 X-Rays Classification}},
  journal = {IEEE Access},
  year = {2021},
  doi = {10.1109/access.2021.3079716},
}
Figure