Leveraging the RETFound foundation model for optic disc segmentation in retinal images

Emanuele Trucco, Frsa, Fiapr, Fbmva, Faaia, Zhenyi Zhao, Muthu Rama Krishnan Mookiah, Emanuele TruccoarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces FunduSegmenter, the first adaptation of the powerful RETFound foundation model for segmenting the optic disc (OD) and optic cup (OC) in retinal fundus images. By introducing novel adapter modules, FunduSegmenter achieves state-of-the-art performance, demonstrating robust generalization capabilities across diverse datasets and challenging conditions, paving the way for more reliable automated retinal image analysis.

What Problem Does This Paper Address?

Accurate segmentation of the optic disc (OD) and optic cup (OC) in retinal fundus images is crucial for diagnosing and monitoring various eye diseases, as these structures serve as key biomarkers. Traditional supervised deep learning methods for this task face several challenges: 1) the laborious and time-consuming nature of annotating medical images, leading to a scarcity of labeled data; 2) a significant drop in performance when models are applied to data from different sources (out-of-distribution), hindering generalization; and 3) the reliance on synthetic data augmentation, which can reduce clinician trust. Foundation models (FMs), trained on vast amounts of unlabeled data, offer a promising solution by learning general representations. However, adapting these FMs, particularly those like RETFound, to specific segmentation tasks in medical imaging, especially for retinal structures, remained an underexplored area.

Methods

The authors propose FunduSegmenter, a novel architecture that adapts the RETFound foundation model for OD and OC segmentation. Key components include:

  • RETFound Encoder: The pre-trained encoder from RETFound, which has learned rich general representations from a massive dataset of retinal images. The weights of this encoder are frozen during training.
  • Segmenter Decoder: A decoder adapted from the Segmenter architecture, known for its effectiveness in natural image segmentation.
  • Novel Adapter Modules:
    • Pre-adapter: A lightweight module designed to handle input images of varying sizes, outperforming simple interpolation methods. It uses an inverted residual architecture inspired by MobileNetV2.
  • Post-adapter: Addresses the challenge of unclear boundaries in OD/OC segmentation by progressively upsampling mask maps, improving convergence speed and accuracy.
  • ViT Block Adapter: Integrated within the RETFound encoder's transformer blocks, this adapter allows for fine-tuning of the encoder's representations for the specific segmentation task while retaining its general features.
  • Skip Connections with CBAM: Feature maps from intermediate layers of the RETFound encoder are passed to the Post-adapter via skip connections, incorporating multi-scale information to improve segmentation accuracy, particularly for the smaller OC.
  • Loss Function: A combination of Dice loss and Cross-Entropy Loss (CELoss) is used to optimize the model.

Datasets

The study utilized a combination of datasets for training and evaluation:

  • Local Dataset: GoDARTS (used for internal verification).
  • Public Datasets:
    • IDRID
    • Drishti-GS
    • RIM-ONE-r3
    • REFUGE

These datasets vary in size and annotation characteristics, allowing for comprehensive evaluation of internal verification, external verification, and domain generalization capabilities.

Key Insights and Ideas

  • Foundation Models for Segmentation: This work demonstrates the significant potential of adapting large foundation models like RETFound for specialized segmentation tasks in medical imaging, moving beyond their original classification applications.
  • Adapter Modules are Key: The introduction of novel adapter modules (Pre-adapter, Post-adapter, ViT Block Adapter) is crucial for effectively fine-tuning a frozen foundation model encoder for segmentation, enabling it to handle variable input sizes and improve segmentation accuracy and stability.
  • Generalization is Achievable: FunduSegmenter shows remarkable stability and generalization across diverse datasets, even those with different distributions or limited data, highlighting the power of pre-trained representations.
  • OD/OC Segmentation Challenges: The paper underscores the specific challenges in OD/OC segmentation, such as unclear boundaries, and how the proposed Post-adapter effectively addresses these.

Key Results

  • State-of-the-Art Performance: FunduSegmenter consistently outperformed six state-of-the-art baseline methods across internal verification, external verification, and domain generalization experiments.
  • High Average DSC: Achieved an average Dice Similarity Coefficient (DSC) of 90.51% in internal verification, significantly outperforming baselines like nnU-Net (82.91%) and DUNet (89.17%).
  • Robust Generalization: Demonstrated strong performance on out-of-distribution datasets, with results only slightly affected by data scarcity or domain shifts, unlike many baseline models.
  • Efficiency: The proposed Pre-adapter is lightweight (0.04M tunable parameters) and supports larger input sizes with faster training times compared to traditional interpolation methods.
  • Improved Stability: The proposed modules helped solve unstable training issues, particularly on small datasets, leading to faster convergence and higher accuracy.

Limitations and Future Work

  • Slightly Behind TVConv in Domain Generalization: While FunduSegmenter performed very well, its results were slightly behind TVConv in domain generalization tasks. However, TVConv has limitations due to its reliance on prior positional knowledge and specific layout assumptions, making it less generalizable.
  • OC Segmentation Performance: While outperforming baselines for OD segmentation, FunduSegmenter's performance for OC segmentation was not as universally dominant, particularly when using REFUGE as the source dataset.
  • Future Work: The authors plan to investigate methods to further improve model performance, especially for OC segmentation. They also aim to design a standard data processing pipeline and expand the model's application to other retinal image analysis tasks (e.g., vessel segmentation) and other image domains. The proposed modules are highlighted as general and extensible to other foundation models.

Summary updated

BibTeX
@misc{faaia2025leveraging,
  author = {Emanuele Trucco, FRSA, FIAPR, FBMVA, FAAIA and Zhenyi Zhao and Muthu Rama Krishnan Mookiah and Emanuele Trucco},
  title = {{Leveraging the RETFound foundation model for optic disc segmentation in retinal images}},
  year = {2025},
  eprint = {2508.11354},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.11354},
}
Figure