AOSLO-net: A deep learning-based method for automatic segmentation of retinal microaneurysms from adaptive optics scanning laser ophthalmoscope images

Qian Zhang, Konstantina Sampani, Mengjia Xu, Shengze Cai, Yixiang Deng, He Li, Jennifer K. Sun, George Em KarniadakisarXiv 2021 · arXiv

Introduction

Diabetic retinopathy (DR) remains a primary cause of vision loss globally. The earliest clinical markers of this condition are microaneurysms (MAs)—small, focal dilatations of retinal capillaries. While traditional fundus photography can identify these lesions as red dots, it lacks the resolution to characterize their complex morphology. Understanding the specific shape and blood flow characteristics of an MA is critical, as certain morphological features are strongly associated with the risk of disease progression.

Adaptive Optics Scanning Laser Ophthalmoscopy (AOSLO) has emerged as a powerful tool, providing cellular-level resolution (~2 $\mu$m). This allows for the visualization of the "neck" and "body" of MAs, as well as the surrounding capillary network. However, the manual segmentation of these features from high-resolution AOSLO videos is labor-intensive and prone to inter-observer variability. To address this, researchers have developed AOSLO-net, a deep learning framework designed for the automated, high-fidelity segmentation of MAs.

The AOSLO-net Architecture

AOSLO-net is built upon a modified U-Net architecture, optimized for the unique challenges of AOSLO imagery, such as low contrast and significant speckle noise.

The AOSLO-net pipeline, including data pre-processing, the EfficientNet-b3 based architecture, and final segmentation output.

The core of the network is a pre-trained EfficientNet-b3 encoder. By leveraging features learned from the massive ImageNet dataset, the model can more effectively identify complex vascular patterns even with a relatively small medical imaging dataset. The decoder follows a standard symmetrical path with skip connections to preserve spatial information.

A key innovation is the multi-modality input. Instead of relying solely on structural images, the model processes two channels:

  1. Enhanced AOSLO Images: These are generated by averaging multiple video frames to reduce noise and highlight static anatomical structures.
  2. Perfusion Maps: These are created by calculating the temporal variance (standard deviation) across video frames. This functional data highlights areas of active blood flow, allowing the model to distinguish between patent MAs and surrounding non-vascular tissue.

Data and Multi-Fidelity Training

The study utilized a dataset of 87 DR AOSLO images from 28 eyes. To ensure the model's robustness against different manual grading styles, the researchers employed a "multi-fidelity" mask approach.

Multi-modality AOSLO inputs and multi-fidelity ground truth masks used for training and evaluation.

Ophthalmologists provided three distinct sets of ground truth masks:

  • Normal: Capturing the true anatomical geometry.
  • Short: Truncating the "neck" or parent vessels.
  • Thick: Exaggerating the width of the vessels.

Training on these variations helped the model learn to prioritize the core structure of the MA while remaining stable regardless of minor variations in how a human expert might define the boundaries.

Results and Clinical Significance

AOSLO-net demonstrated superior performance compared to existing state-of-the-art models. It achieved a mean Dice score of 0.7816, surpassing the widely used nnU-Net (0.7686). More importantly, the model showed high stability, with a lower standard deviation in performance across the test set.

The automated segmentations were used to calculate three vital morphological indices:

  • Largest Caliber (LC): The maximum width of the MA body.
  • Narrowest Caliber (NC): The width of the "neck" connecting the MA to the parent vessel.
  • Body-to-Neck Ratio (BNR): A key metric for classifying MA types.

The high correlation between automated and manual measurements suggests that AOSLO-net can reliably replace manual grading in clinical research, paving the way for large-scale longitudinal studies on diabetic retinopathy progression.

Conclusion

AOSLO-net represents a significant step forward in retinal imaging. By combining structural and functional data within a robust deep learning framework, it overcomes the bottleneck of manual AOSLO analysis. Future work will focus on expanding the dataset and integrating this tool into clinical workflows to better predict patient outcomes and personalize treatment strategies for diabetic retinopathy.

Summary updated

Abstract

Microaneurysms (MAs) are one of the earliest signs of diabetic retinopathy (DR), a frequent complication of diabetes that can lead to visual impairment and blindness. Adaptive optics scanning laser ophthalmoscopy (AOSLO) provides real-time retinal images with resolution down to 2 $μm$ and thus allows detection of the morphologies of individual MAs, a potential marker that might dictate MA pathology and affect the progression of DR. In contrast to the numerous automatic models developed for assessing the number of MAs on fundus photographs, currently there is no high throughput image protocol available for automatic analysis of AOSLO photographs. To address this urgency, we introduce AOSLO-net, a deep neural network framework with customized training policies to automatically segment MAs from AOSLO images. We evaluate the performance of AOSLO-net using 87 DR AOSLO images and our results demonstrate that the proposed model outperforms the state-of-the-art segmentation model both in accuracy and cost and enables correct MA morphological classification.
BibTeX
@misc{zhang2021aoslo,
  author = {Qian Zhang and Konstantina Sampani and Mengjia Xu and Shengze Cai and Yixiang Deng and He Li and Jennifer K. Sun and George Em Karniadakis},
  title = {{AOSLO-net: A deep learning-based method for automatic segmentation of retinal microaneurysms from adaptive optics scanning laser ophthalmoscope images}},
  year = {2021},
  eprint = {2106.02800},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.02800},
}
Figure