Ensemble of Convolutional Neural Networks for Automatic Grading of Diabetic Retinopathy and Macular Edema

Avinash Kori, Sai Saketh Chennamsetty, Mohammed Safwan K. P., Alex VarghesearXiv 2018 · OpenReview

TL;DR

This paper presents an ensemble of pre-trained Convolutional Neural Networks (CNNs) for automatically grading Diabetic Retinopathy (DR) and Diabetic Macular Edema (DME) from fundus images. By leveraging transfer learning and an ensemble approach with model pruning, they achieve high accuracies, addressing the challenge of limited labeled medical data.

What Problem Does This Paper Address?

Diabetic Retinopathy (DR) and Diabetic Macular Edema (DME) are leading causes of blindness, and their severity grading is crucial for timely treatment. Automating this grading process from fundus images is highly desirable. However, a significant challenge in medical image analysis is the scarcity of large, high-quality labeled datasets, which are typically required to train deep learning models like CNNs effectively. This paper aims to overcome this data limitation by developing an automated grading system using CNNs.

Methods

The core of this work lies in an ensemble of Convolutional Neural Networks (CNNs). To address the limited labeled data, the authors employ transfer learning. This involves using CNN models pre-trained on a massive dataset of natural images (ImageNet) and then fine-tuning them on the specific medical imaging tasks of DR and DME grading.

For DR grading, they use a two-tiered approach:

  1. Primary Classifier Ensemble: This ensemble classifies images into four categories: Normal, Mild NPDR, Moderate NPDR, or a combined "Severe NPDR/Proliferative DR" (S-(N)-PDR) class.
  2. Expert Classifier Ensemble: This ensemble is specifically trained to differentiate between Severe NPDR and Proliferative DR (PDR), providing a finer-grained classification when the primary classifier flags an image as S-(N)-PDR.

For DME grading, a "one versus rest" (OVR) approach is used with two ensemble models to handle class imbalance.

A key aspect of their methodology is model pruning. Instead of using all trained models in the ensemble, they select a subset of models that perform well, aiming to improve overall accuracy. Additionally, they utilize a "ten crops" strategy during inference, where multiple cropped versions of an image are fed to the network to capture more information and potentially improve robustness. Predictions from the ensemble are combined using a max-voting scheme to arrive at the final grade.

Datasets

  • Primary Dataset: The paper utilizes a dataset made available as part of the Segmentation and Grading challenge held in conjunction with the 2018 IEEE International Symposium on Biomedical Imaging. This dataset is referred to as the "Indian Diabetic Retinopathy Image Dataset" (IDRiD) in the references.
  • Additional Data: For the "mild" NPDR class in DR grading, where data was limited, they supplemented with a publicly available mild NPDR database (Diaretdb1).

Key Insights and Ideas

  • Ensembling is Powerful: Combining predictions from multiple CNN models significantly boosts performance compared to using a single model, highlighting the benefit of diverse perspectives in classification.
  • Pruning for Performance: Selectively pruning the ensemble (keeping only the best-performing models) can lead to higher accuracy than using all models, suggesting that not all models contribute equally and some might even introduce noise.
  • Expert Models for Nuance: Using a specialized "expert" model to handle challenging distinctions (like differentiating severe NPDR from PDR) can significantly improve overall accuracy, especially when dealing with fine-grained classifications.
  • Data Augmentation via Cropping: The "ten crops" technique, where multiple cropped versions of an image are processed, acts as a form of data augmentation and appears to improve classification accuracy, likely by exposing the model to different spatial views of the pathology.

Key Results

  • DR Grading: The proposed ensemble method achieved an accuracy of 83.9% on the test data (n=56). On the training data, the accuracy was 89.4%.
  • DME Grading: The automated DME grading network achieved an accuracy of 95.45% on the test data (n=44). On the training data, the accuracy was 96.85%.
  • The paper also shows that model pruning improved accuracy by 1.78% compared to using all models in the ensemble for DR grading. The "ten crops" strategy contributed to an accuracy of 85.7% for DR grading.

Limitations and Future Work

  • Limited Test Data Size: The test datasets for both DR (n=56) and DME (n=44) are relatively small, which might limit the generalizability of the results.
  • Expert Model Performance: The paper notes that the expert model for DR grading achieved accuracy below expectations when classifying all 5 classes individually, but it significantly improved the overall ensemble performance by 14% when used to differentiate severe NPDR and PDR.
  • Future Work: The authors suggest that further exploration of ensemble strategies, model pruning techniques, and potentially different network architectures could lead to even better performance. The paper also implies that the "ten crops" strategy is beneficial and could be further optimized.

Summary updated

Abstract

In this manuscript, we automate the procedure of grading of diabetic retinopathy and macular edema from fundus images using an ensemble of convolutional neural networks. The availability of limited amount of labeled data to perform supervised learning was circumvented by using transfer learning approach. The models in the ensemble were pre-trained on a large dataset comprising natural images and were later fine-tuned with the limited data for the task of choice. For an image, the ensemble of classifiers generate multiple predictions, and a max-voting based approach was utilized to attain the final grade of the anomaly in the image. For the task of grading DR, on the test data (n=56), the ensemble achieved an accuracy of 83.9\%, while for the task for grading macular edema the network achieved an accuracy of 95.45% (n=44).
BibTeX
@misc{kori2018ensemble,
  author = {Avinash Kori and Sai Saketh Chennamsetty and Mohammed Safwan K. P. and Alex Varghese},
  title = {{Ensemble of Convolutional Neural Networks for Automatic Grading of Diabetic Retinopathy and Macular Edema}},
  year = {2018},
  eprint = {1809.04228},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1809.04228},
}
Figure