On the Convergence Time of Federated Learning Over Wireless Networks Under Imperfect CSI

Francesco Pase, Marco Giordani, Michele ZorziarXiv 2021 · arXiv

This paper, presented at the IEEE ICC 2021 Workshop, tackles a critical bottleneck in deploying artificial intelligence at the edge: how to train models efficiently over unpredictable wireless links.

TL;DR

By intentionally ignoring model updates from wireless clients with poor connections, researchers found they could speed up Federated Learning convergence by up to 80%. Instead of waiting for every device to finish (the "straggler" problem), their Fixed Rate Federated Learning (FRFL) approach uses channel statistics to set a fast, constant transmission rate that prioritizes training speed over individual update reliability.

What Problem Does This Paper Address?

Federated Learning (FL) allows devices to collaboratively train a shared model without ever uploading their raw, private data to a central server. However, doing this over wireless networks is hard. Standard "Synchronous FL" requires the central base station to wait for every participating client to upload its local update before moving to the next round.

In wireless environments, some clients will inevitably experience "deep fading"—a temporary drop in signal quality. If the system waits for these "stragglers," the entire training process crawls to a halt. Most existing research assumes we have Perfect Channel State Information (CSI)—meaning we know exactly how good the connection is at every millisecond—to optimize transmission. In reality, especially in fast-moving 6G scenarios, perfect CSI is nearly impossible to maintain.

Methods

The authors propose Fixed Rate Federated Learning (FRFL). The core shift is moving from instantaneous optimization to statistical optimization.

  1. Imperfect CSI: Instead of knowing the exact channel state, the base station only knows the Cumulative Distribution Function (CDF)—essentially the "weather report" of how the channel behaves on average.
  2. Fixed Transmission Rate ($R^*$): Rather than asking each client to transmit at its maximum possible rate (which varies wildly), the server sets a single, aggressive global rate.
  3. Embracing Loss: If a client's channel is too weak to support $R^*$, its update is simply dropped for that round.
  4. The Trade-off: While dropping updates sounds bad for accuracy, it allows the system to complete many more training rounds in the same amount of time. The authors use a heuristic to pick $R^*$ such that a target percentage of clients (e.g., 50% or 80%) are likely to succeed.

The study compares this against a baseline Synchronous FL (SFL) that has perfect CSI but must wait for the slowest client in every round.

Datasets

The researchers evaluated their approach using the MNIST dataset, a standard benchmark consisting of 70,000 images of handwritten digits.

  • Model: A Convolutional Neural Network (CNN).
  • Data Distribution: They tested both iid (independent and identically distributed) scenarios, where data is balanced across clients, and non-iid scenarios, which are more realistic and challenging because each device has a different "skewed" subset of data.

Key Insights and Ideas

  • The Straggler Penalty is Massive: As shown in Figure 1, the "average minimum rate" (the speed of the slowest client) drops off a cliff as you add more participants, especially in Rayleigh fading environments. Fig. 1: Average minimum rate/Hz vs. number of participating clients
  • Quantity Over Quality (of Rounds): In a time-constrained environment, it is almost always better to have 50 "okay" rounds where some data is lost than 10 "perfect" rounds where you wait for everyone.
  • Rayleigh vs. the World: The benefits of FRFL are most dramatic in Rayleigh fading (typical for legacy cellular) because it has high variance. In Rician or Nakagami channels (more common in 5G/6G mmWave), the channels are more stable, so the "straggler" problem is less severe, though FRFL still helps.
Fig. 1: Average minimum rate/Hz vs. number of participating clients for the SFL policy in case of Rayleigh, Nakagami and Rician channels and for different values of the quality factor A.

Key Results

  • 80% Time Reduction: In Rayleigh channels, FRFL reached 90% accuracy roughly 80% faster than the baseline SFL.
  • More Rounds, Better Accuracy: Figure 6 illustrates the "aha moment": within a 30-second window, the FRFL approach completed between 32 and 48 rounds, while the standard SFL was stuck at just 12 rounds. This increased frequency allowed the model to reach 95% accuracy while the baseline never got there. Fig. 6: Accuracy vs. number of rounds
  • Robustness to non-iid Data: Even when data was heavily skewed across devices, the speed gains of FRFL outweighed the negative impact of losing occasional updates.
Fig. 6: Accuracy vs. number of rounds

Limitations and Future Work

The primary limitation is the heuristic selection of the fixed rate $R^*$. Currently, the base station picks a rate based on a fixed error threshold (like "let 20% of updates fail"). The authors acknowledge that this might not be the absolute optimal rate.

Future Work includes:

  • Developing learning-based approaches (like Reinforcement Learning) to dynamically adjust the global rate $R^*$ in real-time based on changing network conditions.
  • Exploring how to combine this "rate-fixing" strategy with other techniques like model compression or sparsification to further reduce the wireless burden.

Summary updated

Abstract

Federated learning (FL) has recently emerged as an attractive decentralized solution for wireless networks to collaboratively train a shared model while keeping data localized. As a general approach, existing FL methods tend to assume perfect knowledge of the Channel State Information (CSI) during the training phase, which may not be easy to acquire in case of fast fading channels. Moreover, literature analyses either consider a fixed number of clients participating in the training of the federated model, or simply assume that all clients operate at the maximum achievable rate to transmit model data. In this paper, we fill these gaps by proposing a training process that takes channel statistics as a bias to minimize the convergence time under imperfect CSI. Numerical experiments demonstrate that it is possible to reduce the training time by neglecting model updates from clients that cannot sustain a minimum predefined transmission rate. We also examine the trade-off between number of clients involved in the training process and model accuracy as a function of different fading regimes.
BibTeX
@misc{pase2021convergence,
  author = {Francesco Pase and Marco Giordani and Michele Zorzi},
  title = {{On the Convergence Time of Federated Learning Over Wireless Networks Under Imperfect CSI}},
  year = {2021},
  eprint = {2104.00331},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2104.00331},
}
Figure