In synchronous Federated Learning (FL), a central server coordinates a fleet of edge devices to train a shared machine learning model. The training process operates in discrete rounds: the server broadcasts the current global model, each device computes an update on its local dataset, and the devices upload their updates back to the server for aggregation.
Because the server must wait for all selected participants to submit their updates before updating the global model, the duration of any single training round is dictated by a strict mathematical bottleneck: it is equal to the maximum latency experienced by any individual device. As highlighted by Jianxin Zhao et al. in Parallel and Memory-Efficient Distributed Edge Learning in B5G IoT Networks, distributed edge networks naturally suffer from severe hardware and network variance. When a single device experiences sudden local interference or compute constraints, it becomes a "straggler," forcing high-performance nodes to sit idle and grinding the entire training process to a halt.
Traditional wireless network optimization aims to maximize total network throughput, which typically means allocating more bandwidth to devices with strong signal conditions. In the context of federated learning, however, this strategy is counterproductive. Giving more resources to already-fast devices makes them finish even earlier, widening the idle gap while doing nothing to accelerate the straggler that is actually delaying the round.
To solve this, a prominent research direction focuses on latency equalization: shifting from throughput maximization to a strategy of resource allocation and scheduling designed to synchronize the completion times of all participating nodes.
Inverse Bandwidth Allocation
The core intuition behind latency equalization is to flip traditional resource allocation on its head. Instead of rewarding strong channels, the system allocates communication resources inversely to a device's natural processing speed and channel quality. If a device has a slow processor or a weak wireless signal, the network intentionally grants it a larger share of the available bandwidth to compensate for its deficiencies.
To minimize the latency of a synchronized training round, the system must account for the entire end-to-end pipeline, which includes downloading the model, computing local gradients, and uploading the updates. As demonstrated by Jie Xu et al. in Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks, when bandwidth is distributed such that the sum of these three phases is equalized across all clients, the idle time of the entire system drops to zero.
This inverse allocation strategy is a powerful tool for neutralizing hardware and environmental disparities. Bo Yang et al. show in A Joint Energy and Latency Framework for Transfer Learning over 5G Industrial Edge Networks that providing more communication resources to the weakest links ensures synchronized task completion across highly heterogeneous edge nodes. This mechanism is mathematically formalized through optimization frameworks that adapt to real-time network conditions:
- Min-Max Optimization: Rather than optimizing average performance, the network minimizes the maximum completion time. For example, Su Liu et al. develop a min-max framework in FedAGL: A Communication-Efficient Federated Vehicular Network that allocates bandwidth based on the specific data volume each node needs to transmit, ensuring that the slowest connection does not disproportionately delay the global update.
- Progressive Filling: In massive networks, bandwidth can be distributed using a progressive filling strategy. As explored by Chaoqun You et al. in Hierarchical Personalized Federated Learning Over Massive Mobile Edge Computing Networks, this approach systematically builds up the resource allocation of constrained nodes so they finish their transmissions as simultaneously as possible with high-performance nodes.
- Distance and Signal Awareness: In mobile edge environments, physical distance from the base station dictates channel capacity. The authors of Toward Multiple Federated Learning Services Resource Sharing in Mobile Edge Networks (Minh N. H. Nguyen et al.) mitigate stragglers by dynamically shifting bandwidth to devices with weaker signals or greater physical distances, equalizing the upload times across the spatial layout of the network.
By pairing these communication adjustments with joint optimization of device scheduling—as seen in the semi-synchronous framework proposed by Chaoqun You et al. in Semi-Synchronous Personalized Federated Learning Over Mobile Edge Networks—the network can successfully insulate the global training loop from individual hardware or signal degradation.
Exploiting Asymmetric Slack
When resource allocation is anchored around the timeline of the slowest client, an inherent asymmetry emerges: faster clients possess "slack time." They can complete their baseline tasks quickly and are left with excess computational or communication capacity within the boundaries of the current round. A key facet of straggler mitigation is exploiting this slack to maximize the system's total information gain.
Instead of forcing fast clients to wait idly, the system can dynamically adjust the fidelity of the updates based on available bandwidth. In Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning, Rudan Yan et al. introduce a strategy that uses the estimated upload time of the slowest client as a benchmark for the entire system. While slower clients compress their models heavily to meet the deadline, faster clients utilize their remaining time to transmit lower-compression, highly detailed updates. This ensures that network resources are fully exploited within the constraints of the straggler's latency.
Straggler Timeline: |---- Compute ----|---------- Upload (Heavy Compression) ----------| -> Round Ends
Fast Client Slack: |-- Compute --|-- Upload (Low Compression / High Fidelity) --|...Idle...| -> Round Ends
Conversely, the system can proactively reduce the workload on the stragglers themselves so they can keep pace without requiring an excessive share of network bandwidth. The authors of Adaptive Federated Pruning in Hierarchical Wireless Networks (Shiqiang Wang et al.) assign more aggressive model pruning ratios to devices with poor channel conditions or limited compute power. This resource-aware reduction in model size prevents constrained nodes from becoming bottlenecks.
Similarly, workload balancing can happen on the infrastructure side. In Decentralized Proactive Model Offloading and Resource Allocation for Split and Federated Learning, Binbin Huang et al. proactively allocate additional server-side computational resources to devices with lower local processing power or larger datasets, accelerating their local training phase to match the global schedule.
This temporal alignment can also be managed by staggering the start times of the download phase. Qifan Yan et al. introduce this concept in FedFetch: Faster Federated Learning with Adaptive Downstream Prefetching. By using historical round durations and individual network profiles, the system calculates optimal download start times, allowing slower devices to prefetch model updates in the background with a longer lead time. Consequently, heterogeneous clients complete their updates simultaneously when the training round concludes.
Macro-Level Grouping and Sampling
When the variance in device capabilities is too extreme for fine-grained bandwidth tuning alone, researchers apply macro-level scheduling policies—such as client clustering, partitioning, and latency-aware sampling—to prevent stragglers from disrupting the training loop.
One prominent strategy is to isolate high-variance devices into homogeneous training pools. In Communication Efficiency Optimization of Federated Learning for Computing and Network Convergence of 6G Networks, Yizhuo Cai et al. partition clients into distinct training groups based on their local processing speeds. By ensuring that devices with similar arithmetic power train within the same round, the system minimizes internal latency variance.
This hierarchical approach is mirrored by Guanghao Li et al. in FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity, where devices are clustered into groups based on measured computational latency to guarantee synchronized training cycles within each cluster, preventing high-performance nodes from idling.
Heterogeneous Pool (High Variance):
[Fast Node] ----> Wait...................................... |
[Slow Node] ------------------------------------------------>| -> Global Aggregation Delayed
Partitioned Pools (Low Variance):
Group A: [Fast Node 1] --->|
[Fast Node 2] --->| -> Fast Aggregation
Group B: [Slow Node 1] ------------------------------------->|
[Slow Node 2] ------------------------------------->| -> Slow Aggregation
Alternatively, the system can optimize the client selection process itself to favor efficiency. Bing Luo 0002 et al. demonstrate in Adaptive Heterogeneous Client Sampling for Federated Learning over Wireless Networks that jointly optimizing client sampling probabilities alongside bandwidth allocation ensures that communication bottlenecks do not disproportionately delay the global model update.
To guide this selection intelligently, frameworks can incorporate multi-device scheduling strategies, such as sampling without replacement combined with latency-minimizing bandwidth allocation, as explored by Yinghui He et al. in Scheduling for Cellular Federated Edge Learning With Importance and Channel Awareness.
Taking this a step further, Hongwei Yang et al. design an efficiency-driven selection mechanism in An efficient personalized federated learning approach in heterogeneous environments: a reinforcement learning perspective. The training process incorporates a reward function that explicitly penalizes communication and computation time. This mathematical penalty discourages the selection of slow stragglers unless they provide exceptionally high information gain, effectively balancing statistical progress against wall-clock execution time.
Key Foundational Work
A Joint Energy and Latency Framework for Transfer Learning over 5G Industrial Edge Networks (2021) — contributed 2 concept elements to this direction.
Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks (2021) — contributed 1 concept elements to this direction.
Performance Optimization of Federated Learning over Mobile Wireless Networks (2020) — contributed 1 concept elements to this direction.