The effectiveness of artificial intelligence-based automated grading and training system in education of manual detection of diabetic retinopathy

Qian Xu, Han Jingying, Song Xian, Yuqing Zhao, Lili Wu, Chu Baorui, Wei Guo, Yefeng Zheng, Qıang Zhang, Chunyan Chu, Bian Cheng, Kai Ma, Qu YiFrontiers in Public Health 2022 · DOI

TL;DR

Researchers have developed an AI-driven system that achieves specialist-level accuracy in grading Diabetic Retinopathy (DR) and, more importantly, serves as an effective "robot teacher." By using heatmaps to highlight clinical lesions, the system significantly improved the diagnostic accuracy and sensitivity of medical students and junior residents, proving that AI can bridge the gap between theoretical knowledge and clinical practice.

What Problem Does This Paper Address?

Diabetic Retinopathy (DR) is a leading cause of blindness globally, yet early detection can prevent 90% of severe vision loss. The bottleneck isn't just medicine—it's logistics. There is a critical shortage of retinal specialists, particularly in developing regions, to perform the necessary annual screenings.

While training non-ophthalmologists to screen for DR is a viable solution, traditional medical education struggles to provide enough diverse, standardized cases for students to master the nuances of fundus (back of the eye) photography. This paper explores whether AI can act as a scalable training aid to help novices "see" like experts.

Methods

The authors developed a dual-purpose system for both automated grading and student training:

  • The Architecture: The core is a ResNet-50 image classification network paired with an auxiliary graph convolutional network (GCN). The GCN is used to model the "class-dependency" between different stages of DR (e.g., the progression from mild to moderate), which helps the model avoid underestimating disease severity.
  • Explainability as Education: To make the AI a teacher rather than a "black box," the team used Class Activation Mapping (CAM). This technique generates heatmaps that highlight the specific regions (lesions like hemorrhages or exudates) the AI used to make its diagnosis.
  • The Training Experiment: They recruited junior residents and medical students, putting them through a pre-test, an AI-assisted training phase (where they viewed images alongside AI heatmaps), and a post-test to measure improvement.

Datasets

The study utilized a large-scale, real-world dataset:

  • Source: 78,000 anonymized color fundus images collected from the diabetes clinic of Qilu Hospital of Shandong University between 2016 and 2019.
  • Refinement: After filtering for image quality, they settled on a balanced set of 15,988 images (8,063 DR cases and 7,925 non-DR cases).
  • Ground Truth: Images were graded by three senior ophthalmologists according to the International Clinical Diabetic Retinopathy (ICDR) scale.

Key Insights and Ideas

  • AI as a "Robot Teacher": The most significant takeaway is the shift from AI as a diagnostic tool to AI as an educational catalyst. The system doesn't just give an answer; it provides visual evidence that helps students calibrate their own internal "feature extraction" process.
  • Bridging the Theory-Practice Gap: Students often know what a "cotton-wool spot" is in a textbook, but struggle to find it in a noisy, real-world image. The heatmaps act as a bridge, reinforcing pathological characteristics in situ.
  • Rapid Skill Acquisition: The medical students—the group with the least experience—showed the most dramatic improvements. This suggests that AI-assisted training can significantly shorten the learning curve for complex medical imaging tasks.
  • Sensitivity is Key: The system notably improved the detection of mild DR. In a screening context, missing early-stage disease is a major risk; the AI helped humans become much more "sensitive" to these subtle early markers.

Key Results

  • AI Performance: The automated system achieved an AUC of 0.980 for referable DR (moderate or worse), with a sensitivity and specificity of approximately 96.5%.
  • Human Improvement:
    • Junior Residents: Accuracy improved from 0.947 to 0.978 after AI training.
  • Medical Students: Accuracy jumped from 0.915 to 0.954.
  • Sensitivity Gains: For medical students, sensitivity to mild DR rose from 0.714 to 0.838, a vital improvement for early-stage intervention.
  • Student Feedback: Over 70% of trainees agreed the AI system was helpful and effective, though many noted it should supplement, not replace, traditional instruction.

Limitations and Future Work

  • Missing DME: The study did not include Diabetic Macular Edema (DME), another major cause of vision loss, because the clinic typically uses Optical Coherence Tomography (OCT) rather than fundus images to diagnose it. Future versions would need to integrate DME detection.
  • "Tunnel Vision" Risk: There is a concern that training humans on a DR-specific AI might lead them to overlook other retinal pathologies (like glaucoma or age-related macular degeneration) that a generalist should notice.
  • Sample Size: While the image dataset was large, the number of human trainees was relatively small. The authors suggest larger, multi-institutional studies are needed to validate the system's effectiveness as a standardized part of medical curricula.

Summary updated

Abstract

The AI-based systemdemonstrated high diagnostic accuracy for the detection of DR on fundus images from real-world diabetics, and could be utilized as a training aid system for trainees lacking formal instruction on DR management.
BibTeX
@inproceedings{xu2022effectiveness,
  author = {Qian Xu and Han Jingying and Song Xian and Yuqing Zhao and Lili Wu and Chu Baorui and Wei Guo and Yefeng Zheng and Qıang Zhang and Chunyan Chu and Bian Cheng and Kai Ma and Qu Yi},
  title = {{The effectiveness of artificial intelligence-based automated grading and training system in education of manual detection of diabetic retinopathy}},
  booktitle = {Frontiers in Public Health},
  year = {2022},
  doi = {10.3389/fpubh.2022.1025271},
}
Figure