Research on Classification of COVID-19 Chest X-Ray Image Modal Feature Fusion Based on Deep Learning

Dongsheng Ji, Zhujun Zhang, Yanzhong Zhao, Qianchuan ZhaoJournal of Healthcare Engineering 2021 · DOI

TL;DR

This paper introduces a novel deep learning approach for classifying COVID-19 from chest X-ray images by fusing features from multiple pre-trained convolutional neural networks (CNNs). Their "Model 2," a cascade of five CNNs, significantly outperforms individual models and existing methods, achieving high accuracy, precision, recall, and F1-scores, demonstrating its effectiveness in accurately detecting COVID-19 cases.

What Problem Does This Paper Address?

The rapid spread of COVID-19 necessitates accurate and efficient diagnostic tools. While chest X-rays are a crucial diagnostic aid, traditional image classification models often struggle with low recognition accuracy and fail to fully capture the subtle modal features indicative of COVID-19. This limitation hinders timely and precise diagnosis, especially in resource-constrained settings. The paper aims to address this by developing a more robust and accurate deep learning-based classification system for COVID-19 detection using chest X-rays.

Methods

The proposed method employs a multi-stage approach:

  • Data Augmentation: Chest X-ray images undergo small-sample enhancement techniques like rotation, translation, and random transformations to increase dataset variability and improve model generalization.
  • Feature Extraction: Five established pre-trained CNN models (Xception, ResNet152, DenseNet201, VGG19, and InceptionResNetV2) are utilized to extract modal features from the X-ray images.
  • Feature Fusion: A "cascade network" architecture (specifically "Model 2" in the paper) is proposed, where features extracted by the individual CNNs are concatenated. This fusion aims to leverage the complementary strengths of each model.
  • Parameter Reduction: A global average pooling layer is integrated after the feature extraction stage to reduce training parameters and mitigate overfitting.
  • Classification: The fused features are then passed through a fully connected (FC) layer with three layers (two dense layers with 512 hidden units and a final dense layer with one hidden unit) followed by a sigmoid activation function for binary classification (COVID-19 vs. healthy).
  • Transfer Learning: The study heavily relies on transfer learning, utilizing models pre-trained on the large ImageNet dataset and fine-tuning them on the COVID-19 dataset.

Datasets

The study utilized a publicly available COVID-19 chest X-ray dataset from Kaggle.

  • Training Set: 3278 images
  • Validation (Test) Set: 821 images The dataset exhibits an unbalanced distribution, with a significantly higher number of COVID-19 cases (2389 in training, 598 in validation) compared to normal cases (889 in training, 223 in validation). The authors chose a 0.2 validation split ratio to avoid serious overfitting.

Key Insights and Ideas

  • Feature Fusion is Key: The core idea is that combining features from multiple diverse CNN architectures (Model 2) captures a richer representation of COVID-19 modal characteristics than any single model alone. This ensemble approach effectively overcomes the limitations of individual models in capturing subtle features.
  • Cascade Architecture's Advantage: The "cascade" or ensemble approach, particularly Model 2, demonstrates superior performance. This suggests that parallel processing of features from different models and then combining them is more effective than relying on a single, potentially specialized, model.
  • Small Sample Enhancement: The use of data augmentation techniques is crucial for improving the performance of deep learning models when dealing with limited datasets, as is common with COVID-19 imaging data.
  • Transfer Learning Effectiveness: The study reinforces the power of transfer learning, showing that pre-trained models can be effectively adapted for COVID-19 detection, even with a relatively small dataset.

Key Results

The proposed "Model 2" (a cascade of Xception, ResNet152, DenseNet201, VGG19, and InceptionResNetV2) achieved the best performance:

  • Accuracy: 96.00%
  • Precision: 96.10%
  • Recall: 96.42%
  • F1-score: 95.50%
  • AUC (Area Under the ROC Curve): 0.987

Model 2 outperformed individual CNN models (ResNet50, ResNet152, Xception, InceptionResNetV2) and a simpler cascade model (Model 1). The paper also compares these results to existing works, showing that Model 2 achieves competitive or superior performance across key metrics. For instance, it surpasses the accuracy of Sethy and Behera's ResNet50+SVM (95.33%) and is comparable to Jaiswal et al.'s COVIDPEN (96% accuracy) but with better recall and F1-score.

Limitations and Future Work

  • Dataset Size and Diversity: The study acknowledges the limitation of a relatively small and potentially unbalanced dataset. While data augmentation was used, larger and more diverse datasets would further enhance model robustness.
  • Scope of Classification: The current model is limited to distinguishing between COVID-19 and healthy individuals. It cannot differentiate COVID-19 from other forms of pneumonia or bacterial infections.
  • Image Modality: The research is exclusively based on X-ray images. The authors suggest future work could extend this to CT scans (which are grayscale, unlike RGB X-rays) and explore classification of other lung diseases with similar modal characteristics (e.g., pulmonary edema, lung tumors).
  • Generalizability: While the authors believe the algorithm could be applied to MRI classification, further research is needed to validate this.
  • Future Directions: The authors plan to focus on classifying COVID-19 against bacterial pneumonia and viral pneumonia. They also aim to extend the application to MRI image classification for liver cancer diagnosis.

Summary updated

Abstract

Most detection methods of coronavirus disease 2019 (COVID-19) use classic image classification models, which have problems of low recognition accuracy and inaccurate capture of modal features when detecting chest X-rays of COVID-19. This study proposes a COVID-19 detection method based on image modal feature fusion. This method first performs small-sample enhancement processing on chest X-rays, such as rotation, translation, and random transformation. Five classic pretraining models are used when extracting modal features. A global average pooling layer reduces training parameters and prevents overfitting. The model is trained and fine-tuned, the machine learning evaluation standard is used to evaluate the model, and the receiver operating characteristic (ROC) curve is drawn. Experiments show that compared with the classic model, the classification method in this study can more effectively detect COVID-19 image modal information, and it achieves the expected effect of accurately detecting cases.
BibTeX
@article{ji2021research,
  author = {Dongsheng Ji and Zhujun Zhang and Yanzhong Zhao and Qianchuan Zhao},
  title = {{Research on Classification of COVID-19 Chest X-Ray Image Modal Feature Fusion Based on Deep Learning}},
  journal = {Journal of Healthcare Engineering},
  year = {2021},
  doi = {10.1155/2021/6799202},
}
Figure