FEDI: Few-shot learning based on Earth Mover's Distance algorithm combined with deep residual network to identify diabetic retinopathy

Liangrui Pan, Peng Zhang, Fei Xia, Boya Ji, Wenjuan Liu, Hetian Wang, Yuan Jin, Mitchai Chongcheawchamnan, Shaoliang Peng2021 · DOI

TL;DR

This paper introduces FEDI, a novel few-shot learning approach for identifying diabetic retinopathy (DR) using deep residual networks and Earth Mover's Distance (EMD). By leveraging EMD to measure image similarity and a pre-trained deep residual network, FEDI achieves high accuracy on DR classification with limited training data, addressing the challenge of scarce medical datasets.

What Problem Does This Paper Address?

Diabetic retinopathy (DR) is a leading cause of blindness, and early detection through fundus image analysis is crucial. However, collecting large, annotated datasets of retinal images for training deep learning models is a significant challenge in clinical practice. Traditional deep learning methods require vast amounts of data, making them impractical for rare diseases or specialized medical imaging tasks. This paper aims to bridge this gap by developing a few-shot learning model that can effectively diagnose DR with minimal training samples, thereby reducing the reliance on extensive data collection and potentially improving the objectivity of DR diagnosis.

Methods

The core of FEDI lies in combining two key techniques: a deep residual convolutional neural network (ResNet) for feature extraction and the Earth Mover's Distance (EMD) for similarity comparison.

  1. Deep Residual Network (ResNet): The authors utilize a ResNet architecture to extract robust features from retinal fundus images. ResNets are known for their ability to train very deep networks by using "residual connections" that allow gradients to flow more easily, preventing the vanishing gradient problem. This forms the backbone for learning complex image patterns.
  2. Pre-training: A large-scale pre-training strategy is employed. This involves creating numerous "few-shot learning tasks" from the available dataset. Each task consists of a small "support set" (few examples per class) and a "query set" (examples to be classified). By training on these diverse tasks, the model learns to "learn how to learn" and extract generalizable features, maximizing "experience maximization."
  3. Earth Mover's Distance (EMD): After pre-training, the EMD algorithm is introduced. EMD is a metric learning algorithm that quantifies the "distance" or "cost" of transforming one probability distribution into another. In this context, it's used to calculate the similarity between image feature sets. The paper represents images as sets of local feature vectors and uses EMD to find the optimal matching cost between these sets, thereby capturing nuanced similarities between fundus images.
  4. Fine-tuning with EMD: The pre-trained ResNet's weights are then fine-tuned. The EMD similarity scores are used to calculate the cross-entropy loss, and the model's parameters are updated to improve its classification accuracy. This step leverages the learned features and refines them based on the EMD-based similarity.

Datasets

The primary dataset used in this study is a collection of 1000 retinal fundus images categorized into 39 classes. This dataset is publicly available on Kaggle. The data is split into training, validation, and test sets with a ratio of 0.6:0.2:0.2. The paper also mentions constructing "few-shot classification tasks" (e.g., C-way K-shot tasks) from this dataset for training and evaluation.

Key Insights and Ideas

  • Few-Shot Learning for Medical Imaging: The paper demonstrates that few-shot learning is a viable and effective approach for medical image analysis, particularly when dealing with limited annotated data, a common issue in healthcare.
  • EMD as a Powerful Similarity Metric: Earth Mover's Distance proves to be a valuable tool for capturing subtle similarities between complex image features, going beyond simple pixel-wise comparisons. This is crucial for medical diagnoses where fine details matter.
  • Pre-training for Generalization: The strategy of pre-training on numerous small, diverse tasks allows the model to learn generalizable feature extraction capabilities, which are then fine-tuned for the specific DR classification task.
  • Bridging the Gap: The combination of deep learning for feature extraction and EMD for similarity comparison effectively bridges the gap between raw image data and accurate classification, even with limited examples.

Key Results

The proposed FEDI model achieved a notable 93.5667% accuracy on the 3-way 10-shot task of the diabetic retina test set. The paper also reports that the model can achieve 100% accuracy on the "Normal" class and high accuracy (93.3333%) on classes like "Optic atrophy" and "Retinitis pigmentosa" in the confusion matrix analysis. The pre-training phase itself yielded models with accuracies around 95%.

Limitations and Future Work

  • Pre-training Time: The authors acknowledge that the pre-training process requires a significant amount of time, which might limit the widespread adoption of this method.
  • Future Directions: The paper suggests that this approach offers great potential for AI-assisted treatment of diabetic patients' retina diagnosis, implying that further research and development could lead to more robust and accessible diagnostic tools. The success in achieving high accuracy with limited data opens avenues for applying this methodology to other rare medical conditions or specialized imaging tasks.

Summary updated

Abstract

Diabetic retinopathy(DR) is the main cause of blindness in diabetic patients. However, DR can easily delay the occurrence of blindness through the diagnosis of the fundus. In view of the reality, it is difficult to collect a large amount of diabetic retina data in clinical practice. This paper proposes a few-shot learning model of a deep residual network based on Earth Mover's Distance algorithm to assist in diagnosing DR. We build training and validation classification tasks for few-shot learning based on 39 categories of 1000 sample data, train deep residual networks, and obtain experience maximization pre-training models. Based on the weights of the pre-trained model, the Earth Mover's Distance algorithm calculates the distance between the images, obtains the similarity between the images, and changes the model's parameters to improve the accuracy of the training model. Finally, the experimental construction of the small sample classification task of the test set to optimize the model further, and finally, an accuracy of 93.5667% on the 3wayl0shot task of the diabetic retina test set. For the experimental code and results, please refer to: https://github.com/panliangrui/few-shot-learning-funds.
BibTeX
@inproceedings{pan2021fedi,
  author = {Liangrui Pan and Peng Zhang and Fei Xia and Boya Ji and Wenjuan Liu and Hetian Wang and Yuan Jin and Mitchai Chongcheawchamnan and Shaoliang Peng},
  title = {{FEDI: Few-shot learning based on Earth Mover's Distance algorithm combined with deep residual network to identify diabetic retinopathy}},
  booktitle = {2021},
  year = {2021},
  doi = {10.1109/bibm52615.2021.9669547},
}
Figure