Deep CNN models for predicting COVID-19 in CT and x-ray images

Ahmad Chaddad, Lama Hassan, Christian DesrosiersJournal of Medical Imaging 2021 · DOI

TL;DR

This paper explores the use of deep convolutional neural networks (CNNs) with transfer learning to automatically detect COVID-19 from chest CT and X-ray images. By incorporating specific regions of interest (ROIs) like ground-glass opacities, the models achieved high accuracy, with the DarkNet architecture showing particular promise in distinguishing COVID-19 from other conditions.

What Problem Does This Paper Address?

The COVID-19 pandemic highlighted the urgent need for rapid and reliable diagnostic tools. While medical imaging like CT scans and X-rays can reveal signs of the virus, human interpretation can be subjective and time-consuming, especially with subtle changes. Existing automated methods often struggle with the variability of image findings and the limited availability of labeled data. This paper aims to address this by investigating the effectiveness of deep learning models, specifically leveraging transfer learning, to accurately and efficiently predict COVID-19 infection from chest imaging.

Methods

The core of this research lies in applying Deep Transfer Learning (DTL). The authors utilized six pre-trained CNN architectures (AlexNet, DenseNet, GoogleNet, NASNet-Mobile, ResNet18, and DarkNet) that were originally trained on the massive ImageNet dataset. The idea is that these networks have already learned general image features (like textures and edges) that can be adapted to the new task of identifying COVID-19.

The key novelties in their approach include:

  • Region of Interest (ROI) Integration: For CT scans, they didn't just use the raw images. They also extracted and incorporated specific ROIs known to be indicative of COVID-19: ground-glass opacities (GGO), consolidations, and pleural effusions (PE). These ROIs were segmented from the CT images and fed into the CNNs alongside the original scans.
  • Multi-Modal Approach: The study evaluated the models' performance on both CT scans and X-ray images, and importantly, also explored combining both modalities for prediction.
  • Fine-tuning: For each CNN architecture, the layers following the last convolutional block were replaced with new layers tailored for the COVID-19 classification task. The models were then fine-tuned on the medical imaging datasets.

Datasets

The study utilized several datasets:

  • CT Scans:
    • 349 COVID-19 positive CT images from 216 patients.
  • 397 non-COVID-19 CT images from various sources (Lung Nodule Analysis, MedPix, PubMed Central, Radiopaedia).
  • 100 labeled CT images from 60 COVID-19 patients, specifically segmented to identify GGO, consolidation, and PE findings.
  • X-ray Images:
    • 657 chest X-ray images in total, comprising:
  • 219 COVID-19 positive X-ray images from multiple sources (COVID chest x-ray Dataset, SIRM, Radiopaedia, Radiological Society of North America).
  • 219 normal X-ray images.
  • 219 pneumonia (viral and bacterial) X-ray images from a Kaggle dataset.

The authors did not introduce a new benchmark dataset but rather leveraged existing, publicly available ones and curated a specific set of segmented ROIs.

Key Insights and Ideas

  • ROIs Matter: The inclusion of specific ROIs (GGO, consolidation, PE) significantly improved the performance of several CNN models, particularly when combined. This suggests that focusing on these diagnostically relevant areas can enhance the models' ability to discern COVID-19.
  • DarkNet Shines: Among the tested architectures, DarkNet consistently demonstrated superior performance, especially when integrating ROIs and when combining CT and X-ray data. This highlights the potential of specific CNN architectures for this task.
  • CT vs. X-ray vs. Combined: While both CT and X-ray images yielded good results individually, combining both modalities, particularly with the DarkNet model, led to the highest accuracy and AUC, suggesting a synergistic benefit.
  • Transfer Learning is Viable: The study confirms that transfer learning is a powerful strategy for COVID-19 detection, even with potentially limited medical imaging datasets, by leveraging knowledge from natural image datasets.

Key Results

  • CT Scans (Raw Input): Highest accuracy of 82% and AUC of 88.16%.
  • CT Scans (with ROIs): Incorporating ROIs boosted performance to 82.30% accuracy and 90.10% AUC with the DarkNet model.
  • X-ray Images: Achieved an outstanding AUC of 97% for classifying COVID-19 versus normal versus other conditions.
  • Combined CT and X-ray Images: The DarkNet architecture achieved the highest accuracy of 99.09% and AUC of 99.89% in classifying COVID-19 from non-COVID-19 cases.
  • Confusion Matrix: For X-ray classification (COVID-19, normal, pneumonia), AlexNet and NASNet-Mobile achieved the highest accuracy of 97% for all three classes.

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset Size: While the datasets are substantial, they suggest that including a larger cohort from diverse global regions could lead to a more comprehensive understanding of COVID-19.
  • Limited Patient Data: Clinical demographics such as age, sex, treatments, and overall survival were not available for all cases and thus not considered in the study.
  • Future Directions: The authors propose that future studies could explore additional radiomic markers to assess COVID-19 progression, aiming for improved diagnosis and treatment. They also suggest that these automatic models could reveal further insights into the disease.

Summary updated

Abstract

<b>Purpose:</b> Coronavirus disease 2019 (COVID-19) is a new infection that has spread worldwide and with no automatic model to reliably detect its presence from images. We aim to investigate the potential of deep transfer learning to predict COVID-19 infection using chest computed tomography (CT) and x-ray images. <b>Approach:</b> Regions of interest (ROI) corresponding to ground-glass opacities (GGO), consolidations, and pleural effusions were labeled in 100 axial lung CT images from 60 COVID-19-infected subjects. These segmented regions were then employed as an additional input to six deep convolutional neural network (CNN) architectures (AlexNet, DenseNet, GoogleNet, NASNet-Mobile, ResNet18, and DarkNet), pretrained on natural images, to differentiate between COVID-19 and normal CT images. We also explored the model's ability to classify x-ray images as COVID-19, non-COVID-19 pneumonia, or normal. Performance on test images was measured with global accuracy and area under the receiver operating characteristic curve (AUC). <b>Results:</b> When using raw CT images as input to the tested models, the highest accuracy of 82% and AUC of 88.16% is achieved. Incorporating the three ROIs as an additional model inputs further boosts performance to an accuracy of 82.30% and an AUC of 90.10% (DarkNet). For x-ray images, we obtained an outstanding AUC of 97% for classifying COVID-19 versus normal versus other. Combing chest CT and x-ray images, DarkNet architecture achieves the highest accuracy of 99.09% and AUC of 99.89% in classifying COVID-19 from non-COVID-19. Our results confirm the ability of deep CNNs with transfer learning to predict COVID-19 in both chest CT and x-ray images. <b>Conclusions:</b> The proposed method could help radiologists increase the accuracy of their diagnosis and increase efficiency in COVID-19 management.
BibTeX
@article{chaddad2021deep,
  author = {Ahmad Chaddad and Lama Hassan and Christian Desrosiers},
  title = {{Deep CNN models for predicting COVID-19 in CT and x-ray images}},
  journal = {Journal of Medical Imaging},
  year = {2021},
  doi = {10.1117/1.jmi.8.s1.014502},
}
Figure