Automated deep learning-based AMD detection and staging in real-world OCT datasets (PINNACLE study report 5)

Oliver Leingang, Sophie Riedl, Julia Mai, Gregor S. Reiter, Georg Faustmann, Philipp Fuchs, Hendrik P. N. Scholl, Sobha Sivaprasad, Daniel Rueckert, Andrew Lotery, Ursula Schmidt‐Erfurth, Hrvoje BogunovićScientific 2023 · DOI

TL;DR

Researchers have developed a two-stage deep learning pipeline that automatically detects and stages Age-related Macular Degeneration (AMD) from 3D OCT volumes. By combining 2D slice-level feature extraction with volume-level classification and Monte-Carlo dropout for uncertainty estimation, the model achieves an average ROC-AUC of 0.94 on "messy" real-world clinical data.

What Problem Does This Paper Address?

Age-related Macular Degeneration (AMD) is a leading cause of blindness, and Optical Coherence Tomography (OCT) is the gold standard for diagnosing it. While hospitals are sitting on "treasure troves" of retrospective OCT data, this data is often unstructured and poorly labeled in Electronic Health Records (EHR).

The bottleneck is manual grading: having a retinal specialist sit down and label thousands of 3D scans is prohibitively expensive and slow. Most existing AI models focus on 2D slices (B-scans), but clinical diagnosis requires looking at the entire 3D volume. This paper tackles the challenge of creating a robust, automated pipeline that can accurately stage AMD (Normal, Early/Intermediate, Atrophic, or Neovascular) using the kind of varied-quality scans found in daily clinical practice.

Methods

The authors implemented a clever two-stage modular architecture to avoid the computational heaviness and data requirements of "pure" 3D Convolutional Neural Networks (CNNs):

  1. Stage 1 (B-scan Classifier): An ImageNet-pretrained ResNet50 processes individual 2D slices. It produces a 2048-dimensional latent vector (LatentBscan) for every slice in the volume. This stage provides "spatial explainability" by showing exactly which slices contain disease biomarkers.
  2. Stage 2 (Volume Classifier): The latent vectors from all slices are concatenated into a 2D tensor (LatentVol). This tensor is fed into four separate, smaller ResNets (ResNet18 and ResNet34) that perform binary classification for specific biomarkers: Macular Atrophy (MA), Macular Neovascularization (MNV), Drusen, or Normal.

Uncertainty Estimation: To make the model reliable for clinical use, they employed Monte-Carlo (MC) dropout at inference time. By running the model 10 times per scan with random neurons "dropped out," they can calculate a "Margin of Confidence" (MoC). If the model is unsure, it can flag the scan for human review.

Datasets

  • PINN (Primary): A massive real-world dataset from the UK (Southampton and Moorfields Eye Hospitals) containing 156,872 OCT volumes. A subset of 3,765 scans was used for training.
  • OPT: An in-house dataset from the Medical University of Vienna used to provide "clear-cut" prototype cases.
  • Duke Dataset (External): A public dataset used to test generalization across different hardware (Bioptigen scanners).
  • UK Biobank (External): Used to test the model as a screening tool in a general population with lower-quality scans.

Key Insights and Ideas

  • Modular Flexibility: Because the stages are independent, you can swap out the Stage 1 encoder or retrain the Stage 2 classifier on new biomarkers without starting from scratch.
  • Explainability by Design: By classifying at the B-scan level first, the model naturally produces a "heatmap" of where the disease is located within the 3D volume, helping clinicians verify the results.
  • The "Uncertainty Filter": The authors found that by excluding "uncertain" predictions (using the MoC threshold), the model’s performance on the remaining "high-confidence" subset jumped to near-perfect scores (ROC-AUC > 0.97).
  • Generalization: Despite being trained mostly on Topcon scanners, the model performed exceptionally well on the Duke dataset (Bioptigen scanner), suggesting it has learned fundamental biological biomarkers rather than just scanner-specific artifacts.

Key Results

  • Internal Performance: The model reached an average ROC-AUC of 0.94 on the real-world test set.
  • Specific AUCs: 0.97 for MNV (wet AMD) and 0.96 for MA (atrophy).
  • Drusen detection was slightly lower (0.88), likely due to the difficulty of spotting very small elevations in a single slice.
  • External Validation: On the Duke dataset, the model achieved an almost perfect ROC-AUC of 0.994 for distinguishing AMD from healthy eyes.
  • Efficiency: The pipeline allows for rapid curation of large datasets, turning years of "dark data" in hospital archives into usable research cohorts.

Limitations and Future Work

  • Scanner Specificity: While it generalized to the Duke dataset, the authors note that performance might vary on other popular devices like Heidelberg Spectralis without specific domain adaptation.
  • AMD Focus: The model was trained specifically on AMD patients. In a general screening population, it might struggle with non-AMD pathologies (like diabetic retinopathy) unless specifically trained to recognize them as "Out-of-Distribution."
  • Future Direction: The authors suggest using this tool to automatically label longitudinal datasets to pinpoint the exact "onset" of late-stage AMD, which is critical for clinical trials.

Summary updated

Abstract

Real-world retinal optical coherence tomography (OCT) scans are available in abundance in primary and secondary eye care centres. They contain a wealth of information to be analyzed in retrospective studies. The associated electronic health records alone are often not enough to generate a high-quality dataset for clinical, statistical, and machine learning analysis. We have developed a deep learning-based age-related macular degeneration (AMD) stage classifier, to efficiently identify the first onset of early/intermediate (iAMD), atrophic (GA), and neovascular (nAMD) stage of AMD in retrospective data. We trained a two-stage convolutional neural network to classify macula-centered 3D volumes from Topcon OCT images into 4 classes: Normal, iAMD, GA and nAMD. In the first stage, a 2D ResNet50 is trained to identify the disease categories on the individual OCT B-scans while in the second stage, four smaller models (ResNets) use the concatenated B-scan-wise output from the first stage to classify the entire OCT volume. Classification uncertainty estimates are generated with Monte-Carlo dropout at inference time. The model was trained on a real-world OCT dataset, 3765 scans of 1849 eyes, and extensively evaluated, where it reached an average ROC-AUC of 0.94 in a real-world test set.
BibTeX
@inproceedings{leingang2023automated,
  author = {Oliver Leingang and Sophie Riedl and Julia Mai and Gregor S. Reiter and Georg Faustmann and Philipp Fuchs and Hendrik P. N. Scholl and Sobha Sivaprasad and Daniel Rueckert and Andrew Lotery and Ursula Schmidt‐Erfurth and Hrvoje Bogunović},
  title = {{Automated deep learning-based AMD detection and staging in real-world OCT datasets (PINNACLE study report 5)}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-46626-7},
}
Figure