DR10K: Transfer Learning Using Weak Labels for Grading Diabetic Retinopathy on DR10K Dataset

Mohamed ElHabebe, Shereen Elkordi, Ahmed Gamal-Eldin, Noha Adly, Marwan Torki, Ahmed Elmasry, Islam Sh AhmedWACV 2024 · OpenReview

Abstract

In this paper, we contrast the usage of two deep-learning approaches for the automatic grading of diabetic retinopathy (DR) and diabetic macular edema (DME) in retinal fundus photographs using a relatively small novel dataset. We developed a telemedicine system to collect and humanly grade 11,109 diabetic patients. The certified graders annotated the level of DR as well as the existence of a referable DME in the macula-centered fundus images only. We use EfficientNet to build an AI-based model for both problems. To examine the transfer learning validity, the model was trained on an external dataset (EyePacs) and then finetuned on the Egyptian data for the DR and DME grading problems. Firstly, we use the macula-centered images only in fine-tuning. Secondly, we use optic-disc-centered images in addition to macula-centered images. We obtained the labels for the optic-disc-centered images directly from the corresponding macula-centered labels as weak labels. Then, both types of images are used in fine-tuning. We found an increase in the DR performance using the second approach in both accuracy and quadratic weighted kappa (QWK). Notably, QWK increased from 90.23% to 91.3% using additional weakly labeled optic-disc-centered fundus images.

Introduction

Diabetic Retinopathy (DR) and Diabetic Macular Edema (DME) are leading causes of blindness globally, yet early detection through regular screening can prevent severe vision loss. While deep learning models have shown great promise in automating this screening, they face two major hurdles: domain shift and data scarcity. Models trained on Western datasets often perform poorly when deployed on different ethnicities or hardware. Furthermore, high-quality, human-graded medical images are expensive to produce. This study addresses these gaps by introducing DR10K, a localized Egyptian dataset, and exploring a weak labeling strategy to maximize data utility.

Methods

The researchers employed a multi-stage transfer learning approach using EfficientNet as their primary architecture. They compared two configurations for the network's classification head to optimize performance on the grading task.

Proposed EfficientNet-based network architecture showing the two configurations (A and B) tested for classification.

The core technical novelty lies in the Weak Labeling strategy. In a typical clinical exam, multiple views of the eye are taken: one centered on the macula and one on the optic disc. Usually, only macula-centered images are graded. The authors hypothesized that disease state is consistent across the eye and "propagated" labels from macula images to unlabeled optic-disc images of the same eye, effectively doubling the training data.

The multi-stage pipeline for training on external data (Kaggle), fine-tuning on local data, and final model evaluation.

Additionally, they used Regression Score Binary Thresholding. Instead of simple classification, the model predicts a continuous value. By tuning the threshold on this scale, they could better balance sensitivity and specificity—critical for medical screening where missing a sick patient is a high-risk failure.

Datasets

  • Kaggle (EyePacs/Aptos): Used for initial pre-training (92,000+ images).
  • Messidor-2: Used as an external benchmark to prove generalization.
  • DR10K: Collected via a telemedicine system in Egypt, consisting of 11,109 patient exams and 19,378 gradable images, including both macula and optic-disc views.

Key Results

  • DR Grading: The best ensemble model achieved 89.86% accuracy and a 91.3% QWK on the 5-class DR scale.
  • DME Detection: For detecting referable Macular Edema, the model reached an AUC of 98.92%.
  • Impact of Weak Labels: Adding weakly labeled optic-disc images improved the QWK from 90.23% to 91.3%, suggesting that different perspectives of the same pathology help the model learn more robust features.
  • Clinical Utility: Fine-tuning on local data proved essential; models trained solely on international datasets saw a significant performance drop when tested on the Egyptian population.

Conclusion

The DR10K study demonstrates that local context and weak labeling strategies can significantly boost the reliability of automated DR screening. By leveraging existing but unlabeled clinical images, researchers can improve model robustness without increasing the burden on human graders. Future work will focus on expanding the dataset and integrating longitudinal patient data.

Summary updated

Abstract

In this paper, we contrast the usage of two deep-learning approaches for the automatic grading of diabetic retinopathy (DR) and diabetic macular edema (DME) in retinal fundus photographs using a relatively small novel dataset. We developed a telemedicine system to collect and humanly grade 11,109 diabetic patients. The certified graders annotated the level of DR as well as the existence of a referable DME in the macula-centered fundus images only. We use EfficientNet to build an AI-based model for both problems. To examine the transfer learning validity, the model was trained on an external dataset (EyePacs) and then finetuned on the egyptian data for the DR and DME grading problems. Firstly, we use the macula-centered images only in fine-tuning. Secondly, we use optic-disc-centered images in addition to macula-centered images. We obtained the labels for the optic-disc-centered images directly from the corresponding macula-centered labels as weak labels. Then, both types of images are used in fine-tuning. We found an increase in the DR performance using the second approach in both accuracy and quadratic weighted kappa(QWK). Notably, QWK increased from 90.23% to 91.3% using additional weakly labeled optic-disc-centered fundus images.
BibTeX
@inproceedings{elhabebe2024dr,
  author = {Mohamed ElHabebe and Shereen Elkordi and Ahmed Gamal-Eldin and Noha Adly and Marwan Torki and Ahmed Elmasry and Islam SH Ahmed},
  title = {{DR10K: Transfer Learning Using Weak Labels for Grading Diabetic Retinopathy on DR10K Dataset}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
  url = {https://openreview.net/forum?id=gKprMFTfhJ},
}
Figure