Abstract
Deep learning-based techniques have been widely utilized for brain tumor segmentation using both single and multi-modal Magnetic Resonance Imaging (MRI) images. Most current studies focus on centralized training due to the intrinsic challenge of data sharing across clinics. To mitigate privacy concerns, researchers have introduced Federated Learning (FL) methods to brain tumor segmentation tasks. However, currently such methods are focusing on single modal MRI, with limited study on multi-modal MRI. The challenges include complex structure, large-scale parameters, and overfitting issues of the FL based methods using multi-modal MRI. To address the above challenges, we propose a novel multi-modal FL framework for brain tumor segmentation (Fed-MUnet) that is suitable for FL training. We evaluate our approach with the BraTS2022 datasets, which are publicly available. The experimental results demonstrate that our framework achieves FL nature of distributed learning and privacy preserving. For the enhancing tumor, tumor core and whole tumor, the mean of five major metrics were 87.5%, 90.6% and 92.2%, respectively, which were higher than SOTA methods while preserving privacy. In terms of parameters count, quantity of floating-point operations (FLOPs) and inference, Fed-MUnet is Pareto optimal compared with the state-of-the-art segmentation backbone while achieves higher performance and tackles privacy issue.
Introduction
Brain tumor segmentation is a critical task in medical imaging, where multi-modal MRI data (T1, T1c, T2, and FLAIR) provides complementary information necessary for accurate diagnosis. However, the centralization of such sensitive data is often prohibited by privacy regulations like HIPAA and GDPR. Federated Learning (FL) offers a promising alternative by allowing models to be trained locally at clinical centers, sharing only model updates rather than raw data.
Despite its potential, applying FL to multi-modal MRI segmentation remains challenging. Existing multi-modal models are often too complex, featuring massive parameter counts that lead to overfitting and high communication overhead in decentralized environments. Consequently, most current FL research in this domain has been limited to single-modal data. Fed-MUnet addresses these limitations by introducing a lightweight, one-stage framework specifically optimized for multi-modal federated training.
Methods
The Fed-MUnet framework centers on a novel segmentation backbone, M-Unet, designed for efficiency and robustness. Unlike traditional multi-modal models that may require complex two-stage pipelines, M-Unet is an end-to-end architecture. It utilizes a symmetric U-Net structure where the encoder employs Convolutional Neural Networks (CNNs) for efficient local feature extraction and downsampling.
To effectively fuse the four MRI modalities, the authors introduce a Cross Modality Module (CMM). This module leverages Transformer-based self-attention to capture long-range dependencies across modalities, overcoming the local receptive field limitations of standard CNNs. By stacking and flattening multi-modal features into a 2D matrix before processing, the CMM ensures a comprehensive integration of spatial and modal information.
Privacy is maintained through DP-FedAvg, which incorporates gradient clipping and the addition of Gaussian noise to the aggregated model weights. The model was trained and validated using the BraTS2022-GLI dataset, with data distributed across simulated clients using a Dirichlet distribution to represent clinical data variability.
Results and Discussion
Fed-MUnet achieves state-of-the-art performance while strictly adhering to privacy constraints. When evaluated on the BraTS2022 dataset with differential privacy noise, the model yielded Dice scores of 0.860 for Enhancing Tumor (ET), 0.894 for Tumor Core (TC), and 0.916 for Whole Tumor (WT). These results demonstrate that privacy-preserving measures can be implemented with minimal impact on segmentation accuracy.
A key advantage of Fed-MUnet is its efficiency. It is Pareto optimal in terms of complexity, requiring only 71.61 million parameters and 13.98 GFLOPs. This is significantly lower than competing models like MultiCNN (177.31M parameters), making Fed-MUnet better suited for the communication-constrained environments typical of Federated Learning. The model's inference time of 0.186 seconds further underscores its suitability for real-world clinical deployment.
Conclusion
Fed-MUnet represents a significant step forward in privacy-preserving medical image analysis. By combining efficient CNN-based feature extraction with Transformer-based cross-modality fusion, the framework achieves high-accuracy brain tumor segmentation without the overhead of traditional multi-modal models. Future research will focus on addressing extreme data imbalances across clients and exploring more robust fusion techniques to handle the inherent heterogeneity of MRI data across different clinical sites.
Summary updated