Abstract
Federated machine learning is a versatile tool to utilize distributed data, but traditional methods struggle with heterogeneous data and task distributions, which are common in healthcare. We propose Partial Meta-Federated Learning (PMFL), integrating meta-learning with partial parameter sharing to address this. Evaluated on real-world medical datasets, PMFL achieves superior performance and faster training speeds compared to standard federated learning.
Introduction
Standard Federated Learning (FL), such as the widely used FedAvg algorithm, typically assumes that data is "homogeneous"—meaning every client (e.g., a hospital) is trying to solve the same task using similar data distributions. In the real world, especially in healthcare, this is rarely true. One hospital might have a dataset focused on pneumonia, while another focuses on lung lesions. This "task heterogeneity" causes standard FL to perform poorly because the global model tries to find a single average set of weights that doesn't actually fit any specific task well.
Methodology
The authors propose two advancements to bridge the gap between federated learning and diverse real-world tasks:
Meta-Federated Learning (MetaFL)
MetaFL integrates the Model-Agnostic Meta-Learning (MAML) framework into FL. Instead of clients sending model gradients or weights to the server, they communicate the training loss from their local epochs. The server aggregates these losses to optimize the global model's adaptability, effectively training it to be a "great starter" that can quickly adapt to any client's specific task.
Partial Meta-Federated Learning (PMFL)
To further improve generalization and prevent "catastrophic forgetting," PMFL introduces partial parameter sharing. Inspired by sparsification techniques in continual learning, the server only updates a subset of the global model parameters (e.g., half the nodes) while keeping the rest frozen. This "half-frozen" approach allows the model to balance global feature extraction with local task adaptation, ensuring the global model doesn't drift too far toward any single client's distribution.
Experimental Evaluation
The researchers evaluated PMFL on two major real-world medical datasets:
- MIMIC-CXR v2.0.0: A massive dataset of chest X-ray radiology reports with 8 distinct disease labels (e.g., Pneumonia, Atelectasis) simulating heterogeneous tasks.
- eICU Collaborative Research Database: Granular critical care data from over 200 hospitals, used to create patient representations for 7 different disease labels.
Results and Discussion
PMFL consistently outperformed all baselines in accuracy, stability, and speed across both datasets.
- Superior Accuracy: In the MIMIC-CXR "Pleural Other" task, PMFL reached an AUC of 0.9799, compared to just 0.7166 for standard FL and 0.8702 for a model trained without any federated pre-training.
- Convergence Speed: PMFL converged much faster than baselines—often within just 4 epochs—and showed significantly lower variance across different experiments.
- Mitigating Negative Transfer: On the eICU dataset, where tasks were even more divergent, standard FL actually performed worse than no FL at all (0.8774 vs 0.9218). PMFL corrected this "malignant" interference, reaching an AUC of 0.9562.
The success of PMFL highlights the power of using "loss as a signal" for adaptability and the importance of parameter freezing as a regularizer in diverse environments.
Conclusion
PMFL provides a robust solution for federated learning in heterogeneous environments. By combining the adaptability of meta-learning with the stability of partial parameter sharing, it enables effective collaborative training across diverse medical tasks without compromising privacy or performance.
Summary updated