Abstract: Recent advancements in neuroimaging have led to greater data sharing among the scientific community. However, institutions frequently maintain control over their data, citing concerns related to research culture, privacy, and accountability. This creates a demand for innovative tools capable of analyzing amalgamated datasets without the need to transfer actual data between entities. To address this challenge, we propose a decentralized sparse federated learning (FL) strategy. This approach emphasizes local training of sparse models to facilitate efficient communication within such frameworks. By capitalizing on model sparsity and selectively sharing parameters between client sites during the training phase, our method significantly lowers communication overheads. This advantage becomes increasingly pronounced when dealing with larger models and accommodating the diverse resource capabilities of various sites. We demonstrate the effectiveness of our approach through the application to the Adolescent Brain Cognitive Development (ABCD) dataset.
Introduction
Neuroimaging research increasingly relies on large, multi-site datasets to achieve the statistical power necessary for meaningful discovery. Projects like the Adolescent Brain Cognitive Development (ABCD) study provide unprecedented scale, yet they also present significant logistical hurdles. Moving raw MRI data between institutions is often prohibited by strict privacy regulations, such as HIPAA or GDPR, and the sheer volume of high-resolution imaging files makes centralized storage costly and slow.
Federated Learning (FL) has emerged as a potential solution, allowing institutions to train models locally and share only model updates (weights) with a central server. However, standard FL faces two primary bottlenecks: the massive communication overhead required to transmit millions of parameters over standard internet connections, and the performance degradation caused by non-Identically and Independently Distributed (non-IID) data across different clinical sites. To overcome these barriers, we introduce NeuroSFL (Sparse Federated Learning for Neuro-Imaging).
Methodology: Sparse Federated Learning
The core innovation of NeuroSFL lies in its decentralized sparse training strategy, which identifies and optimizes only the most "important" sub-networks within a model from the start of the training process.
1. Sub-network Discovery via SNIP
Instead of training a full, dense model, NeuroSFL utilizes the Single-shot Network Pruning (SNIP) criterion. At the beginning of the process, each site calculates the sensitivity of the loss function to specific neural connections using a small batch of local data. This identifies which parameters are most critical for the task at hand.
2. Global Masking
While some methods allow sites to develop their own local masks, this often leads to a "densification" problem when the server averages divergent updates. NeuroSFL aggregates local importance scores to create a single global mask. This shared blueprint ensures that all sites are optimizing the same sparse structure, preserving the communication advantage throughout the entire training lifecycle.
3. Communication Efficiency
By training only the parameters within the global mask, the system significantly reduces the amount of data transmitted. Using Compressed Sparse Row (CSR) encoding, NeuroSFL only sends non-zero values, allowing institutions with limited bandwidth to participate in large-scale collaborative AI research without significant delays.
Experimental Setup
We evaluated NeuroSFL using two primary benchmarks:
- ABCD Dataset: T1-weighted MRI scans from 11,875 participants (aged 9-10) were used for a gender classification task to test performance on high-dimensional neuroimaging data.
- CIFAR-10: This dataset was used to measure the scalability of the method across various architectures, specifically ResNet32 through ResNet110, to quantify "wall-time" communication speedups.
Key Results and Insights
High Accuracy at Extreme Sparsity
NeuroSFL demonstrated remarkable resilience to pruning. On the ABCD dataset, the model maintained 71.18% accuracy even at 95% sparsity. In comparison, competing methods like IndividualSNIP saw performance plummet to 52.70% at the same sparsity level. This highlights the importance of the global mask in maintaining model integrity.
Sparsity as a Regularizer
One of the most surprising findings was that at 50% sparsity, NeuroSFL (92.4% accuracy) actually outperformed the dense FedAvg baseline (90.5%). This suggests that by pruning redundant or noisy connections, the sparse approach acts as a form of regularization, helping the model generalize better across diverse imaging sites.
Real-World Speedups
In practical testing using the COINSTAC system—a distributed neuroimaging platform—NeuroSFL achieved a 2.32x speedup for ResNet110 models compared to standard dense federated learning. This efficiency makes the framework particularly suitable for large-scale models where communication time often exceeds computation time.
Robustness to Data Imbalance
The method proved robust to non-IID data. In simulations where some sites had as few as 5 samples while others had over 5,000, the global model maintained high accuracy and convergence stability, addressing a major pain point in multi-hospital collaborations.
Conclusion and Future Work
NeuroSFL provides a scalable and efficient framework for distributed neuroimaging, significantly reducing the communication barrier while maintaining high predictive performance. While this study focused on gender classification, future research will explore the application of sparse FL to more complex tasks, such as predicting cognitive scores or identifying early-stage disease biomarkers. Additionally, further tuning of hyperparameter sensitivity and learning rate decay will be essential to optimize the trade-off between communication speed and final model accuracy in diverse clinical settings.
Summary updated