Introduction
Federated Learning (FL) has emerged as a powerful paradigm for training machine learning models across decentralized mobile devices while preserving data privacy. However, deploying FL in real-world mobile environments presents significant challenges due to system heterogeneity. Mobile devices vary drastically in hardware capabilities (e.g., high-end GPUs vs. low-power CPUs) and network conditions (e.g., fluctuating Wi-Fi signal strength). These factors lead to the "straggler problem," where slow devices delay the entire global training round.
Existing approaches like HeteroFL attempt to mitigate this by assigning smaller "subnetworks" to weaker devices. However, these assignments are typically static and ignore the dynamic nature of both system resources and the training process itself. This paper introduces WHALE-FL, a framework that employs Adaptive Subnetwork Scheduling to dynamically optimize training efficiency based on real-time system conditions and model importance.
Methodology: Adaptive Subnetwork Scheduling
WHALE-FL replaces static subnetwork assignments with a dynamic mechanism driven by a novel Utility Function. This function balances two critical factors:
- System Efficiency (SE): This metric estimates the completion time for a device based on its current transmission rate and available computational power.
- Training Efficiency (TE): This measures the potential contribution of a device's update to the global model. WHALE-FL utilizes Fisher Information (FedFIM) to quantify this. High Fisher Information indicates that the model is in a "Critical Learning Period," requiring larger subnetwork updates to capture essential features.
The Workflow
At the start of each training round, devices calculate their utility based on current hardware/network status and recent Fisher Information. This utility is mapped to a specific subnetwork size (e.g., 1/4th or 1/16th of the full model width). The server then aggregates these multi-scale updates using a specialized averaging technique to ensure parameter fairness across the global model.
Experimental Evaluation
The researchers evaluated WHALE-FL across four diverse tasks: Image Classification (MNIST, CIFAR-10), Language Modeling (WikiText2), and Human Activity Recognition (HAR). The experiments were conducted on a physical testbed of 20 heterogeneous devices, including MacBook Pros, NVIDIA Jetsons, and Raspberry Pi 4s.
Key Results
WHALE-FL consistently outperformed both standard FL (FedAvg) and state-of-the-art subnetwork methods like HeteroFL.
- Training Speedup: WHALE-FL achieved target accuracy 1.3x to 2.1x faster than FedAvg and up to 1.74x faster than HeteroFL. By "right-sizing" the task for each device, it effectively mitigated the impact of stragglers.
- Improved Accuracy: Beyond speed, WHALE-FL often reached higher final accuracy levels. For instance, on the MNIST dataset, it improved final accuracy to 89.29%, compared to 87.42% for HeteroFL.
- Resilience to Non-IID Data: The framework showed significant gains in statistically diverse data scenarios, proving its robustness for real-world mobile deployments.
Conclusion
WHALE-FL addresses the core inefficiencies of mobile Federated Learning by making subnetwork scheduling both system-aware and training-aware. By using Fisher Information as a "learning compass," it ensures that computational resources are concentrated where they are most needed during critical training phases. Future work will explore combining width-wise scaling with depth-wise scaling and further optimizing the hyperparameter tuning process for the utility function.
Summary updated