Reliable Joint Segmentation of Retinal Edema Lesions in OCT Images

Meng Wang, Kai Yu, Chun-Mei Feng, Ke Zou, Yanyu Xu, Qingquan Meng, Rick Siow Mong Goh, Yong Liu, Xinxing Xu, Huazhu Fu

TL;DR

This paper introduces a novel deep learning model for accurately and reliably segmenting multiple types of retinal edema lesions in OCT images. By combining wavelet-enhanced feature extraction with a multi-scale transformer and an uncertainty-aware segmentation head, the model achieves state-of-the-art performance while also providing a confidence score for its predictions, crucial for clinical applications.

What Problem Does This Paper Address?

Diabetic retinopathy (DR) is a leading cause of vision loss, and its diagnosis often relies on analyzing Optical Coherence Tomography (OCT) images to identify retinal edema lesions. These lesions, including Retinal Edema (RE), Sub-retinal Fluid (SRF), and Pigment Epithelial Detachment (PED), present significant segmentation challenges due to blurred boundaries, varying scales, and complex interdependencies. Existing segmentation methods, primarily based on Convolutional Neural Networks (CNNs), struggle with capturing long-range dependencies and can lose fine details of small lesions like PEDs during down-sampling. Furthermore, a critical gap exists in assessing the reliability of these segmentation results, which is essential for clinical decision-making. This paper aims to address these limitations by developing a robust and uncertainty-aware segmentation framework.

Methods

The proposed method, a "reliable multi-scale wavelet-enhanced transformer network," builds upon a U-shaped architecture and comprises four key components:

  • Wavelet-Enhanced Feature Extractor: This module replaces the standard down-sampling operations in a pre-trained ResNet with an Adaptive Wavelet Down-Sampling (AWDS) module. AWDS uses a 2D adaptive lifting scheme to perform multi-resolution wavelet transforms, generating sub-bands that preserve fine details and avoid information loss, especially for small lesions.
  • Adaptive Multi-Scale Transformer (AMsTrans) Module: Appended to the feature extractor, AMsTrans is designed to capture multi-scale, long-range dependencies. It takes feature maps from different encoder levels, normalizes them, and processes them through scale-specific dot-product attention blocks. An adaptive fusion mechanism then combines these features, inspired by artificial neurons, to better represent global contextual information across various lesion scales.
  • Decoder: This standard component gradually restores spatial information and fuses multi-semantic contextual features from the encoder.
  • Uncertainty-Aware Segmentation Head: This novel component, based on subjective logic evidential theory, generates not only the segmentation masks but also an uncertainty score map. This map highlights regions where the model has low confidence, indicating potential errors and the need for physician review.

Datasets

The primary dataset used for training and evaluation is the AI-Challenge 2018 for retinal edema lesions segmentation. This dataset contains OCT cubes with ground truth annotations for RE, SRF, and PED. The paper notes a significant imbalance in lesion proportions, with PED being particularly sparse, posing a challenge for segmentation. The dataset was split into training (6:2:2 ratio), validation, and testing sets.

Key Insights and Ideas

  • Wavelets for Detail Preservation: The integration of wavelet transforms is crucial for preserving fine structural details in OCT images, particularly for small lesions like PEDs, which are easily lost in traditional down-sampling operations.
  • Multi-Scale Transformers for Global Context: Capturing long-range dependencies across different scales is essential for understanding the complex morphology of retinal edema lesions. AMsTrans effectively addresses this by processing features from multiple encoder levels.
  • Uncertainty for Reliability: Providing an explicit uncertainty score alongside segmentation masks is a significant step towards clinical applicability. This allows for a more trustworthy assessment of segmentation quality and helps identify areas requiring human verification.
  • Addressing Data Imbalance: The paper acknowledges the challenge of imbalanced datasets (e.g., sparse PEDs) and demonstrates that their proposed method, particularly with the wavelet enhancement, performs well even on these difficult cases.

Key Results

The proposed method achieved state-of-the-art performance on the AI-Challenge 2018 dataset, outperforming various CNN-based, transformer-based, and uncertainty-based methods.

  • Quantitative Improvements: The paper reports significant improvements in Jaccard index, Dice similarity index, and Accuracy across all lesion types (RE, SRF, PED). For instance, the Jaccard index for PED lesions was improved by 25.8% compared to a baseline U-Net with a ResNet backbone. The average Jaccard, Dice, and Accuracy achieved were 0.821, 0.896, and 0.970, respectively.
  • Robustness: The method demonstrated improved robustness to noise, maintaining higher segmentation performance than other methods when OCT images were degraded with Gaussian noise.
  • Qualitative Improvements: Visual results showed a significant alleviation of over-segmentation and missing-segmentation issues, especially for small PED lesions. The uncertainty maps effectively highlighted potentially erroneous segmentation regions.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Dataset Diversity: The current evaluation is primarily on a single dataset. Future work should involve testing on datasets with greater variance in data distribution, such as those collected from different OCT scanners and acquisition modes, to further verify performance.
  • Semi-Supervised Learning: Annotating pixel-level labels for retinal edema lesions is time-consuming and expensive. The authors propose exploring semi/weakly supervised methods to leverage large amounts of unlabeled data to further improve segmentation performance.
  • Complex Medical Image Segmentation: The authors plan to continue working on adapting and improving their proposed method for more complex medical image segmentation tasks beyond retinal edema.

Summary updated

Abstract

Focusing on the complicated pathological features, such as blurred boundaries, severe scale differences between symptoms, background noise interference, etc., in the task of retinal edema lesions joint segmentation from OCT images and enabling the segmentation results more reliable. In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network, which can provide accurate segmentation results with reliability assessment. Specifically, aiming at improving the model's ability to learn the complex pathological features of retinal edema lesions in OCT images, we develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module of our newly designed. Meanwhile, to make the segmentation results more reliable, a novel uncertainty segmentation head based on the subjective logical evidential theory is introduced to generate the final segmentation results with a corresponding overall uncertainty evaluation score map. We conduct comprehensive experiments on the public database of AI-Challenge 2018 for retinal edema lesions segmentation, and the results show that our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches. The code will be released on: https://github.com/LooKing9218/ReliableRESeg.
BibTeX
@misc{wang2022reliable,
  author = {Meng Wang and Kai Yu and Chun-Mei Feng and Ke Zou and Yanyu Xu and Qingquan Meng and Rick Siow Mong Goh and Yong Liu and Xinxing Xu and Huazhu Fu},
  title = {{Reliable Joint Segmentation of Retinal Edema Lesions in OCT Images}},
  year = {2022},
  eprint = {2212.00330},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2212.00330},
}
Figure