Introduction
Deep learning has revolutionized medical imaging, particularly in surgical tasks such as polyp detection and skin lesion segmentation. However, the development of robust models is often hindered by two significant challenges: strict data privacy regulations (e.g., GDPR) and "domain shift"—the performance degradation that occurs when a model trained on data from one hospital is applied to another with different imaging protocols or hardware. This paper proposes a novel federated learning (FL) framework designed to navigate these hurdles by decoupling the model architecture between a central server and local clients.
The Problem: Privacy and Domain Heterogeneity
Standard deep learning requires centralized data, which is often impossible in healthcare due to legal and ethical constraints. While Federated Learning (FL) traditionally addresses this by training models locally and aggregating updates (e.g., FedAvg), it often struggles with non-Identically and Independently Distributed (non-IID) data. In surgical imaging, differences in lighting, camera angles, and equipment noise across institutions create a significant domain gap that a single "average" global model cannot easily bridge.
Methodology: The Client-Server Architecture
The authors introduce a specialized FL framework based on an encoder-decoder structure. The core innovation lies in the distribution of these components:
1. The Source Phase
The process begins with a "source phase" where a complete model is trained on a large, well-annotated public dataset. This establishes a baseline understanding of the target anatomy.
2. The Immutable Server-Side Decoder
Following the source phase, the decoder—the part of the network responsible for generating the final segmentation mask—is uploaded to a central server and frozen. This "immutable" decoder serves as a global anchor, representing the universal knowledge of what a specific feature (like a polyp) looks like.
3. Local Client Encoders
Each participating hospital (client) maintains its own local encoder. During training, the client processes its unique images through this encoder to produce a latent representation—a compressed, abstract version of the data. This latent vector is the only information sent to the server.
4. Domain Adaptation
The server's frozen decoder processes the latent vector and calculates the loss, which is sent back to the client. The client then updates only its local encoder. This forces the encoder to learn how to map its domain-specific images into a latent space that the universal decoder can interpret, effectively "translating" local data into a global format.
Experimental Evaluation
The framework was rigorously tested across two distinct medical imaging domains:
- Endoscopic Polyp Segmentation: Using Kvasir-SEG as the source, the model was adapted to EndoUDA, CVC-ClinicDB, and ETIS-Larib datasets. These represent a wide variety of imaging conditions and hardware.
- Skin Lesion Segmentation: Using the ISIC dataset as the source and PH2 as the target domain.
Key Insights
- Architectural Flexibility: The server only requires a fixed latent dimension. This allows different hospitals to use encoders of varying sizes (Small, Medium, or Large) based on their local computational resources.
- Privacy Preservation: By transmitting abstract latent vectors rather than raw pixels, the framework significantly reduces the risk of sensitive data exposure. The authors compare this to public-key cryptography, where the encoder acts as a private key and the decoder as a public key.
- Stability: Freezing the decoder prevents "parameter poisoning" and catastrophic forgetting, ensuring that the model remains anchored to the core task.
Results and Performance
The proposed method, particularly the variant initialized with source weights (FtEn), demonstrated superior performance compared to traditional FL methods:
- Outperforming FedAvg: In the ETIS-Larib polyp dataset, the method achieved a 69.9 mIoU, showing greater consistency across diverse datasets where standard FedAvg and FedDG often fluctuated.
- Superiority over Independent Training: The framework consistently outperformed models trained solely on local data (INDP), which often suffer from overfitting due to small sample sizes. This proves the value of "borrowing" the decoder's knowledge from a larger source.
- Resource Efficiency: The system maintained high effectiveness even when clients utilized smaller, less computationally expensive encoders, making it a viable solution for smaller clinics.
Limitations and Future Directions
While promising, the authors identify several areas for further research:
- Latent Space Security: Although safer than raw data, latent vectors may still be vulnerable to "inversion attacks." Future iterations could incorporate encryption layers.
- Structural Constraints: The requirement for a fixed latent dimension necessitates agreement on network bottlenecks across all participating sites.
- Communication Overhead: While efficient, the real-world bandwidth requirements for frequent latent vector transmission need further empirical study.
Conclusion
This client-server federated learning approach offers a robust solution for cross-domain surgical image segmentation. By keeping encoders local and decoders central and immutable, the framework effectively handles domain shift while maintaining high standards of data privacy and architectural flexibility.
Summary updated