Federated Self-supervised Domain Generalization for Label-efficient Polyp Segmentation

Xinyi Tan, Jiacheng Wang, Liansheng WangarXiv 2025 · DOI · arXiv

Abstract

Deep learning models for polyp segmentation face a "triple threat" in clinical practice: label scarcity, data privacy silos, and domain shift. This paper introduces LFDG (Label-efficient Federated Domain Generalization), a framework that enables accurate segmentation across medical centers without sharing private patient data or requiring massive labeled datasets. By combining federated self-supervised learning with a novel adversarial data augmentation strategy (SSADA) and a consistency-preserving guardrail (SRAM), LFDG achieves significant performance gains and superior generalization to entirely unseen hospital domains.

Introduction

Clinical deployment of automated polyp segmentation is often hindered by the high cost of expert manual annotation and strict data privacy laws (like HIPAA or GDPR) that prevent hospitals from pooling images. Furthermore, a model trained at one site often fails when deployed at another due to "domain shift"—variations in imaging equipment, lighting, and patient demographics.

While Federated Learning (FL) and Self-supervised Learning (SSL) address privacy and label scarcity respectively, their combination often struggles to generalize to new, "unseen" hospitals. LFDG bridges this gap by simulating domain diversity during local training, ensuring that models remain robust even when faced with diverse geographical and technical shifts.

Methodology

The LFDG framework integrates three sophisticated components to maximize both label efficiency and domain robustness:

  1. Federated Self-supervised Pre-training: Using the DropPos method (a Vision Transformer-based SSL technique), the model learns to reconstruct image patches and predict their positions. This allows the network to learn the underlying "anatomy" of colonoscopy images without needing any manual labels.
  2. Self-supervised Adversarial Data Augmentation (SSADA): To prepare for unseen domains, the system enters a "min-max" adversarial game. It intentionally perturbs images to maximize reconstruction loss, creating "hard" examples that simulate potential domain shifts. The model then learns to minimize this loss, effectively stress-testing itself against worst-case scenarios.
  3. Source-reconstruction and Augmentation-masking (SRAM): To prevent adversarial training from distorting images into unrecognizable noise, SRAM acts as a "guardrail." It uses a masked autoencoder approach to ensure that even the most perturbed images maintain semantic consistency with the original source, preventing the model from learning from "hallucinated" features.
The LFDG framework showing the interaction between local client training (SSADA/SRAM) and global server aggregation.

Experimental Results

The researchers evaluated LFDG using the PolypGen multi-center dataset for training and the CVC-ClinicDB dataset as a completely "unknown domain" to test zero-shot generalization capabilities.

Quantitative Performance

LFDG consistently outperformed both standard federated learning baselines and state-of-the-art self-supervised methods:

  • PolypGen C1: LFDG achieved a Mean IoU of 62.83%, outperforming the FedDropPos baseline (59.03%) and other popular methods like FedMAE and FedBEiT.
  • Generalization: On the completely unseen CVC-ClinicDB domain, LFDG maintained a lead with a 55.87% Mean IoU, proving its robustness against domain shift.

Qualitative Analysis

Visual comparisons demonstrate that LFDG produces much tighter, more accurate segmentation masks compared to other SSL methods, which often struggle with boundary definition or fragmented predictions.

Visual comparison of segmentation results. LFDG (Ours) shows much tighter, more accurate masks compared to other SSL methods.

Ablation Studies

Ablation experiments confirmed the necessity of each component: removing the SSADA component dropped performance by 2.50%, and removing the SRAM module dropped it by 1.30%. This validates that both the adversarial "challenge" and the semantic "guardrail" are essential for success.

Discussion and Conclusion

LFDG demonstrates that adversarial training can serve as a powerful proxy for data diversity in federated settings where global data is inaccessible. The study also reinforces the robustness of Vision Transformers (ViT) over traditional CNNs in handling heterogeneous medical data distributions.

While promising, the framework's iterative "min-max" training introduces additional computational overhead for local clients. Furthermore, performance is sensitive to the SRAM balancing parameter ($\beta$), which may require tuning for different medical tasks. Future work will explore extending these generalizable representations to polyp classification and detection tasks.

Summary updated

Abstract

Employing self-supervised learning (SSL) methodologies assumes par-amount significance in handling unlabeled polyp datasets when building deep learning-based automatic polyp segmentation models. However, the intricate privacy dynamics surrounding medical data often preclude seamless data sharing among disparate medical centers. Federated learning (FL) emerges as a formidable solution to this privacy conundrum, yet within the realm of FL, optimizing model generalization stands as a pressing imperative. Robust generalization capabilities are imperative to ensure the model's efficacy across diverse geographical domains post-training on localized client datasets. In this paper, a Federated self-supervised Domain Generalization method is proposed to enhance the generalization capacity of federated and Label-efficient intestinal polyp segmentation, named LFDG. Based on a classical SSL method, DropPos, LFDG proposes an adversarial learning-based data augmentation method (SSADA) to enhance the data diversity. LFDG further proposes a relaxation module based on Source-reconstruction and Augmentation-masking (SRAM) to maintain stability in feature learning. We have validated LFDG on polyp images from six medical centers. The performance of our method achieves 3.80% and 3.92% better than the baseline and other recent FL methods and SSL methods, respectively.
BibTeX
@misc{tan2025federated,
  author = {Xinyi Tan and Jiacheng Wang and Liansheng Wang},
  title = {{Federated Self-supervised Domain Generalization for Label-efficient Polyp Segmentation}},
  year = {2025},
  eprint = {2502.07951},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.07951},
}
Figure