Abstract
Federated learning (FL) is a promising approach for collaborative medical imaging without sharing raw data. However, clinical practice often results in "missing modalities" across different institutions, creating significant intermodal heterogeneity. We introduce FedMEPD, a novel framework that addresses this by employing modality-specific encoders, a partially personalized fusion decoder, and Localized Adaptive Calibration via Cross-Attention (LACCA). By leveraging global feature anchors, FedMEPD enables clients with incomplete data to "hallucinate" missing information. Experimental results on BraTS and HaN-Seg benchmarks demonstrate that FedMEPD significantly outperforms state-of-the-art FL methods, improving mean Dice Similarity Coefficient (mDSC) by up to 12% compared to standard FedAvg.
Introduction
In medical imaging, particularly brain tumor segmentation, a full suite of MRI sequences (T1, T1c, T2, and FLAIR) is ideal. However, real-world clinical protocols vary significantly due to cost, time, or lack of contrast agents, leading to missing modalities at different sites. This intermodal heterogeneity poses a challenge for standard Federated Learning (FL), which typically assumes uniform data types across all participants. Furthermore, institutions often require models personalized to their specific local data distributions rather than a generic global model.
Methodology
FedMEPD (Federated Modality-specific Encoders and Partially Personalized Fusion Decoder) is built upon three core components designed to handle missing data and promote effective knowledge sharing:
- Federated Modality-specific Encoders: Instead of a monolithic encoder, FedMEPD uses four distinct encoders tailored to each MRI sequence. Clients only train and synchronize encoders for the modalities they possess, allowing for parameter specialization.
- Partially Personalized Fusion Decoder: This component merges features from the encoders. It uses a consistency-based mask to decide which parameters to federate. If a filter's local updates align with global trends, it is shared; otherwise, it becomes personalized to the local client. A "patience" hyperparameter ($P$) ensures that personalization only occurs after consistent divergence, preventing overreaction to noise.
- Multi-Anchor Calibration (LACCA): The server generates "anchors" — representative feature prototypes for tumor regions like edema or the core. Clients use Localized Adaptive Calibration via Cross-Attention (LACCA) to align their incomplete features with these global anchors, effectively compensating for missing modality information.
Experimental Results
The framework was evaluated on the BraTS 2018/2020 and HaN-Seg datasets. FedMEPD consistently outperformed existing methods:
- Accuracy Boost: On BraTS 2018, it achieved a 5% mDSC improvement over FedMSplit and a 12% improvement over standard FedAvg.
- Server Performance: The global server model, benefiting from specialized client knowledge, reached an mDSC of 84.98%, surpassing models trained solely on local server data.
- Robustness: The system maintained high performance even with limited server data (10%) or degraded label quality.
Discussion and Conclusion
FedMEPD effectively bridges the gap caused by missing modalities in federated environments. The use of feature anchors provides a privacy-preserving mechanism to share global knowledge without exposing raw data. While the current model assumes balanced client data, future work will focus on dynamic weight assignment for varying client sizes and further optimizing communication overhead. FedMEPD represents a significant step toward robust, personalized federated learning in heterogeneous clinical settings.
Summary updated