RetiFluidNet: A Self-Adaptive and Multi-Attention Deep Convolutional Network for Retinal OCT Fluid Segmentation

Reza Rasti, Armin Biglari, Mohammad Rezapourian, Ziyun Yang, Sina Farsiu

Abstract

Optical coherence tomography (OCT) helps ophthalmologists assess macular edema, accumulation of fluids, and lesions at microscopic resolution. Quantification of retinal fluids is necessary for OCT-guided treatment management, which relies on a precise image segmentation step. As manual analysis of retinal fluids is a time-consuming, subjective, and error-prone task, there is increasing demand for fast and robust automatic solutions. In this study, a new convolutional neural architecture named RetiFluidNet is proposed for multi-class retinal fluid segmentation. The model benefits from hierarchical representation learning of textural, contextual, and edge features using a new self-adaptive dual-attention (SDA) module, multiple self-adaptive attention-based skip connections (SASC), and a novel multi-scale deep self-supervision learning (DSL) scheme. The attention mechanism in the proposed SDA module enables the model to automatically extract deformation-aware representations at different levels, and the introduced SASC paths further consider spatial-channel interdependencies for concatenation of counterpart encoder and decoder units, which improve representational capability. RetiFluidNet is also optimized using a joint loss function comprising a weighted version of dice overlap and edge-preserved connectivity-based losses, where several hierarchical stages of multi-scale local losses are integrated into the optimization process. The model is validated based on three publicly available datasets: RETOUCH, OPTIMA, and DUKE, with comparisons against several baselines. Experimental results on the datasets prove the effectiveness of the proposed model in retinal OCT fluid segmentation and reveal that the suggested method is more effective than existing state-of-the-art fluid segmentation algorithms in adapting to retinal OCT scans recorded by various image scanning instruments.

Introduction

Optical Coherence Tomography (OCT) has become the gold standard for imaging the retina, providing microscopic resolution that allows clinicians to visualize distinct retinal layers. In pathologies such as Diabetic Macular Edema (DME) and Age-related Macular Degeneration (AMD), fluid accumulates within or beneath these layers. The precise quantification of these fluids—specifically Intraretinal Fluid (IRF), Subretinal Fluid (SRF), and Pigment Epithelial Detachment (PED)—is critical for guiding treatment and monitoring disease progression.

Despite its importance, manual segmentation of retinal fluids is a laborious, subjective, and error-prone process. The task is further complicated by the inherent characteristics of OCT images, including speckle noise, low contrast, and the high variability of fluid pocket morphology across different patients and imaging hardware. To address these challenges, this paper presents RetiFluidNet, a novel deep learning architecture designed for automated, multi-class retinal fluid segmentation.

Methodology

RetiFluidNet is an encoder-decoder architecture that incorporates several advanced modules to enhance feature extraction and spatial coherence.

Self-Adaptive Dual-Attention (SDA) Module

The SDA module is a key innovation that allows the network to dynamically adjust its focus. By utilizing two trainable parameters, $\alpha$ and $\beta$, the module adaptively weights spatial attention (focusing on the location of fluid) and channel attention (focusing on the most relevant feature maps). This mechanism enables the model to extract deformation-aware representations that are robust to the varying shapes and sizes of retinal lesions.

Self-Adaptive Attention-based Skip Connections (SASC)

Standard skip connections in U-Net architectures can sometimes transfer irrelevant or noisy information from the encoder to the decoder. RetiFluidNet addresses this by integrating the SDA module into the skip connections. These SASC paths filter the encoder features, ensuring that only the most relevant spatial and channel information is used for reconstruction in the decoder, thereby improving the precision of the final segmentation map.

Residual Multi-Kernel Pooling (RMP)

To effectively segment fluid pockets ranging from small micro-cysts to large detachments, RetiFluidNet employs Residual Multi-Kernel Pooling. This module allows the network to process information at multiple scales simultaneously, ensuring that both local details and global context are preserved.

Deep Self-Supervision Learning (DSL) and Optimization

The model is trained using a multi-scale Deep Self-Supervision Learning (DSL) strategy. Auxiliary loss functions are applied at multiple internal stages of the decoder, forcing the network to learn robust, hierarchical representations. The final optimization uses a joint loss function that combines a weighted Dice overlap loss with an edge-preserved connectivity-based loss (Bicon). This approach ensures that the segmented regions are not only accurate in terms of pixel count but also spatially coherent with well-defined boundaries.

Experimental Results and Discussion

RetiFluidNet was validated on three public benchmarks: RETOUCH, OPTIMA, and DUKE.

Performance Benchmarks

On the RETOUCH dataset, which includes scans from three different manufacturers (Cirrus, Spectralis, and Topcon), RetiFluidNet achieved a mean Dice Similarity Coefficient (DSC) of 86.6% and a Balanced Accuracy of 94.9%. These results represent a significant improvement over existing state-of-the-art models, such as MsTGANet, which it outperformed by 3.1% in DSC.

Robustness Across Instruments

A major hurdle in clinical AI is the "domain shift" between different imaging devices. RetiFluidNet demonstrated exceptional stability across different OCT scanners. This instrument-invariant performance suggests that the model's self-adaptive mechanisms effectively normalize features across varying image qualities and noise profiles.

Ablation Analysis

Ablation studies highlighted the critical role of the connectivity-based loss. When this component was removed, performance for certain fluid types dropped by as much as 6.1%. This underscores the importance of incorporating spatial connectivity constraints into the learning process for medical image segmentation.

Conclusion

RetiFluidNet offers a robust and efficient solution for the automated segmentation of retinal fluids in OCT scans. By combining self-adaptive attention mechanisms with multi-scale self-supervision and connectivity-aware loss functions, the model achieves state-of-the-art performance and demonstrates high reliability across different imaging platforms. While the current 2D implementation is highly effective for clinical use, future research will focus on extending the architecture to 3D volumetric data to further enhance its diagnostic capabilities.

Summary updated

Abstract

Optical coherence tomography (OCT) helps ophthalmologists assess macular edema, accumulation of fluids, and lesions at microscopic resolution. Quantification of retinal fluids is necessary for OCT-guided treatment management, which relies on a precise image segmentation step. As manual analysis of retinal fluids is a time-consuming, subjective, and error-prone task, there is increasing demand for fast and robust automatic solutions. In this study, a new convolutional neural architecture named RetiFluidNet is proposed for multi-class retinal fluid segmentation. The model benefits from hierarchical representation learning of textural, contextual, and edge features using a new self-adaptive dual-attention (SDA) module, multiple self-adaptive attention-based skip connections (SASC), and a novel multi-scale deep self supervision learning (DSL) scheme. The attention mechanism in the proposed SDA module enables the model to automatically extract deformation-aware representations at different levels, and the introduced SASC paths further consider spatial-channel interdependencies for concatenation of counterpart encoder and decoder units, which improve representational capability. RetiFluidNet is also optimized using a joint loss function comprising a weighted version of dice overlap and edge-preserved connectivity-based losses, where several hierarchical stages of multi-scale local losses are integrated into the optimization process. The model is validated based on three publicly available datasets: RETOUCH, OPTIMA, and DUKE, with comparisons against several baselines. Experimental results on the datasets prove the effectiveness of the proposed model in retinal OCT fluid segmentation and reveal that the suggested method is more effective than existing state-of-the-art fluid segmentation algorithms in adapting to retinal OCT scans recorded by various image scanning instruments.
BibTeX
@article{rasti2023retifluidnet,
  author = {Reza Rasti and Armin Biglari and Mohammad Rezapourian and Ziyun Yang and Sina Farsiu},
  title = {{RetiFluidNet: A Self-Adaptive and Multi-Attention Deep Convolutional Network for Retinal OCT Fluid Segmentation}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2023},
  doi = {10.1109/tmi.2022.3228285},
}
Figure