WHALE-FL: Wireless and Heterogeneity Aware Latency Efficient Federated Learning over Mobile Devices via Adaptive Subnetwork Scheduling

Huai-An Su, Jiaxiang Geng, Liang Li, Xiaoqi Qin, Yanzhao Hou, Hao Wang, Xin Fu, Miao Pan

Introduction

Federated Learning (FL) has emerged as a powerful paradigm for training machine learning models across decentralized mobile devices while preserving data privacy. However, deploying FL in real-world mobile environments presents significant challenges due to system heterogeneity. Mobile devices vary drastically in hardware capabilities (e.g., high-end GPUs vs. low-power CPUs) and network conditions (e.g., fluctuating Wi-Fi signal strength). These factors lead to the "straggler problem," where slow devices delay the entire global training round.

Existing approaches like HeteroFL attempt to mitigate this by assigning smaller "subnetworks" to weaker devices. However, these assignments are typically static and ignore the dynamic nature of both system resources and the training process itself. This paper introduces WHALE-FL, a framework that employs Adaptive Subnetwork Scheduling to dynamically optimize training efficiency based on real-time system conditions and model importance.

Methodology: Adaptive Subnetwork Scheduling

WHALE-FL replaces static subnetwork assignments with a dynamic mechanism driven by a novel Utility Function. This function balances two critical factors:

  1. System Efficiency (SE): This metric estimates the completion time for a device based on its current transmission rate and available computational power.
  2. Training Efficiency (TE): This measures the potential contribution of a device's update to the global model. WHALE-FL utilizes Fisher Information (FedFIM) to quantify this. High Fisher Information indicates that the model is in a "Critical Learning Period," requiring larger subnetwork updates to capture essential features.

The Workflow

At the start of each training round, devices calculate their utility based on current hardware/network status and recent Fisher Information. This utility is mapped to a specific subnetwork size (e.g., 1/4th or 1/16th of the full model width). The server then aggregates these multi-scale updates using a specialized averaging technique to ensure parameter fairness across the global model.

Figure 4: The relationship between Local Fisher Information and the adaptive subnetwork size level (p) over training time, illustrating how WHALE-FL scales model complexity based on learning importance.

Experimental Evaluation

The researchers evaluated WHALE-FL across four diverse tasks: Image Classification (MNIST, CIFAR-10), Language Modeling (WikiText2), and Human Activity Recognition (HAR). The experiments were conducted on a physical testbed of 20 heterogeneous devices, including MacBook Pros, NVIDIA Jetsons, and Raspberry Pi 4s.

Key Results

WHALE-FL consistently outperformed both standard FL (FedAvg) and state-of-the-art subnetwork methods like HeteroFL.

  • Training Speedup: WHALE-FL achieved target accuracy 1.3x to 2.1x faster than FedAvg and up to 1.74x faster than HeteroFL. By "right-sizing" the task for each device, it effectively mitigated the impact of stragglers.
  • Improved Accuracy: Beyond speed, WHALE-FL often reached higher final accuracy levels. For instance, on the MNIST dataset, it improved final accuracy to 89.29%, compared to 87.42% for HeteroFL.
  • Resilience to Non-IID Data: The framework showed significant gains in statistically diverse data scenarios, proving its robustness for real-world mobile deployments.
Figure 2: Testing Accuracy vs. Training Time, demonstrating that WHALE-FL reaches higher accuracy significantly faster than baseline methods.

Conclusion

WHALE-FL addresses the core inefficiencies of mobile Federated Learning by making subnetwork scheduling both system-aware and training-aware. By using Fisher Information as a "learning compass," it ensures that computational resources are concentrated where they are most needed during critical training phases. Future work will explore combining width-wise scaling with depth-wise scaling and further optimizing the hyperparameter tuning process for the utility function.

Summary updated

Abstract

As a popular distributed learning paradigm, federated learning (FL) over mobile devices fosters numerous applications, while their practical deployment is hindered by participating devices' computing and communication heterogeneity. Some pioneering research efforts proposed to extract subnetworks from the global model, and assign as large a subnetwork as possible to the device for local training based on its full computing and communications capacity. Although such fixed size subnetwork assignment enables FL training over heterogeneous mobile devices, it is unaware of (i) the dynamic changes of devices' communication and computing conditions and (ii) FL training progress and its dynamic requirements of local training contributions, both of which may cause very long FL training delay. Motivated by those dynamics, in this paper, we develop a wireless and heterogeneity aware latency efficient FL (WHALE-FL) approach to accelerate FL training through adaptive subnetwork scheduling. Instead of sticking to the fixed size subnetwork, WHALE-FL introduces a novel subnetwork selection utility function to capture device and FL training dynamics, and guides the mobile device to adaptively select the subnetwork size for local training based on (a) its computing and communication capacity, (b) its dynamic computing and/or communication conditions, and (c) FL training status and its corresponding requirements for local training contributions. Our evaluation shows that, compared with peer designs, WHALE-FL effectively accelerates FL training without sacrificing learning accuracy.
BibTeX
@inproceedings{su2025whale,
  author = {Huai-An Su and Jiaxiang Geng and Liang Li and Xiaoqi Qin and Yanzhao Hou and Hao Wang and Xin Fu and Miao Pan},
  title = {{WHALE-FL: Wireless and Heterogeneity Aware Latency Efficient Federated Learning over Mobile Devices via Adaptive Subnetwork Scheduling}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i19.34272},
}
Figure