Update Aware Device Scheduling for Federated Learning at the Wireless Edge

Mohammad Mohammadi Amiri, Deniz Gündüz, Sanjeev R. Kulkarni, H. Vincent Poor

Introduction

Federated Learning (FL) has emerged as a powerful paradigm for training machine learning models across distributed edge devices—such as smartphones and IoT sensors—without the need to centralize raw data. However, deploying FL at the wireless edge introduces a significant challenge: the communication bottleneck. In environments with limited bandwidth, a fundamental trade-off arises between the number of participating devices and the precision of their model updates. This paper investigates "update-aware" scheduling policies that optimize this trade-off by considering both wireless channel conditions and the informational importance of local updates.

The Communication-Learning Trade-off

In a typical FL round, devices compute local gradients and transmit them to a central server for aggregation. When many devices share a limited wireless channel, the bandwidth allocated to each device shrinks. To fit within this "bit-budget," devices must employ lossy compression, such as sparsification and quantization.

Most existing FL literature assumes perfect communication or ignores the physical layer. This paper argues that such assumptions are unrealistic. The authors identify a "Quality vs. Quantity" dilemma: Is it more beneficial to aggregate many noisy, heavily compressed updates, or a few high-quality, high-fidelity updates?

Methodology: Update-Aware Scheduling

The researchers propose a digital transmission framework utilizing Digital Stochastic Gradient Descent (D-SGD). In this scheme, model updates are sparsified—keeping only the most significant values—and then quantized. To manage the wireless resource, the paper introduces four distinct scheduling policies:

  1. Best Channel (BC): A traditional approach that schedules devices based solely on the strongest wireless signal-to-noise ratio.
  2. Best $l_2$-norm (BN2): A significance-based approach that prioritizes devices whose model updates have the largest magnitude, representing the most substantial changes to the model.
  3. BC-BN2 (Hybrid): A two-stage policy that first filters for devices with adequate channel quality and then selects those with the most significant updates.
  4. BN2-C (Channel-Aware Significance): The most advanced policy, which evaluates the significance of an update after accounting for the compression necessitated by the device's specific wireless channel.

Supporting these policies is a rigorous convergence analysis, proving that the global model will reach an optimal solution even under the constraints of lossy compression and selective scheduling.

Experimental Results and Insights

The framework was evaluated using the MNIST handwritten digit dataset under both IID (Independent and Identically Distributed) and Non-IID data distributions.

The "Single Device" Phenomenon in IID Data

In IID scenarios, where every device has a representative mix of data, the authors found that scheduling only a single device ($K=1$) per round often performs best. Because the data is homogeneous, one high-precision update provides more value to the global model than ten noisy, heavily compressed updates. In these tests, the BN2-C policy reached 93.1% accuracy, significantly outperforming the channel-only (BC) approach at 91.2%.

Diversity in Non-IID Environments

When data is Non-IID (e.g., one device only sees '1's and '2's), the single-device strategy fails. The model becomes unstable, oscillating toward the bias of whichever device was last selected. In these cases, scheduling a moderate number of devices is necessary to ensure the global model receives a diverse, representative gradient. The proposed hybrid policies (BC-BN2 and BN2-C) maintained stability here, achieving over 81% accuracy, while simpler methods struggled at 77-78%.

Significance vs. Signal Quality

The research highlights a temporal shift in priority during the training process:

  • Early Stage: When the model is far from convergence, gradients are large. Prioritizing update significance (BN2) is most effective for making rapid progress.
  • Late Stage: As the model nears convergence, updates become smaller and more sensitive to noise. At this point, channel quality (BC) becomes critical to ensure that subtle fine-tuning information is not lost during transmission.

Conclusion and Future Directions

By integrating awareness of both the physical wireless layer and the importance of the learning updates, the proposed scheduling policies offer a more efficient path to convergence for FL at the edge. While the BN2-C policy introduces some computational overhead for devices to estimate post-compression significance, the gains in accuracy and stability are substantial.

Future research may explore adaptive scheduling—where the number of scheduled devices ($K$) changes dynamically as the model converges—and the integration of dynamic power allocation to further optimize energy efficiency on battery-constrained devices.

Summary updated

Abstract

We study federated learning (FL) at the wireless edge, where power-limited devices with local datasets collaboratively train a joint model with the help of a remote parameter server (PS). We assume that the devices are connected to the PS through a bandwidth-limited shared wireless channel. At each iteration of FL, a subset of the devices are scheduled to transmit their local model updates to the PS over orthogonal channel resources, while each participating device must compress its model update to accommodate to its link capacity. We design novel scheduling and resource allocation policies that decide on the subset of the devices to transmit at each round, and how the resources should be allocated among the participating devices, not only based on their channel conditions, but also on the significance of their local model updates. We then establish convergence of a wireless FL algorithm with device scheduling, where devices have limited capacity to convey their messages. The results of numerical experiments show that the proposed scheduling policy, based on both the channel conditions and the significance of the local model updates, provides a better long-term performance than scheduling policies based only on either of the two metrics individually. Furthermore, we observe that when the data is independent and identically distributed (i.i.d.) across devices, selecting a single device at each round provides the best performance, while when the data distribution is non-i.i.d., scheduling multiple devices at each round improves the performance. This observation is verified by the convergence result, which shows that the number of scheduled devices should increase for a less diverse and more biased data distribution.
BibTeX
@inproceedings{amiri2020update,
  author = {Mohammad Mohammadi Amiri and Deniz Gündüz and Sanjeev R. Kulkarni and H. Vincent Poor},
  title = {{Update Aware Device Scheduling for Federated Learning at the Wireless Edge}},
  booktitle = {ISIT},
  year = {2020},
  doi = {10.1109/isit44484.2020.9173960},
}
Figure