Joint Device Selection and Power Control for Wireless Federated Learning

Wei Guo, Ran Li, Chuan Huang, Xiaoqi Qin, Kaiming Shen, Wei Zhang

TL;DR

This paper introduces a novel framework for wireless Federated Learning (FL) that jointly optimizes which devices participate and how much power they use for communication. By using analog "over-the-air computation" (AirComp) and a clever adaptive reweighing scheme, the authors minimize the learning error caused by noisy wireless channels, achieving performance remarkably close to ideal, error-free scenarios.

What Problem Does This Paper Address?

Federated Learning (FL) allows multiple devices (like smartphones or IoT sensors) to train a shared machine learning model without ever sharing their raw data. However, implementing FL over wireless networks is hard. Most existing research assumes "perfect" digital communication, but in reality, wireless channels are noisy, limited in bandwidth, and vary over time.

Digital communication often leads to high latency as the number of devices grows. To solve this, researchers have looked at AirComp (Over-the-Air Computation), where devices transmit analog signals simultaneously, and the wireless channel itself performs the "summation" needed for model aggregation. While fast, AirComp is sensitive to channel noise and fading. This paper tackles the challenge of how to select the best devices and control their transmit power to ensure the model still converges accurately despite these wireless imperfections in both the downlink (server-to-device) and uplink (device-to-server).

Methods

The authors propose a joint device selection and power control scheme based on three key pillars:

  1. Analog-to-Analog Pipeline: Unlike digital FL, the Parameter Server (PS) broadcasts the model in an analog fashion, and devices upload updates via AirComp. This bypasses the "cliff effect" of digital modulation and reduces communication overhead.
  2. Adaptive Reweighing: Standard FL (FedAvg) uses fixed weights based on data size. This paper proposes an AirComp-based adaptive reweighing scheme where the aggregation weights are directly determined by the devices' uplink transmit power. This allows the system to "down-weight" devices with poor channel conditions automatically.
  3. Optimality Gap Minimization: The authors derive a mathematical upper bound for the "optimality gap" (the difference between the current model's loss and the theoretical minimum). They then formulate an optimization problem to minimize this gap.
  4. Semidefinite Relaxation (SDR): The optimization problem is a "mixed-integer" nightmare (picking devices is a 0/1 choice). The authors transform it into a continuous problem (QCRQ) and solve it efficiently using the SDR technique, which provides a high-quality approximate solution.

Datasets

The researchers evaluated their algorithm using two standard computer vision benchmarks:

  • MNIST: A dataset of 70,000 handwritten digits.
  • CIFAR-10: A more complex dataset of 60,000 color images across 10 classes (airplanes, cats, etc.).

Both datasets were tested in i.i.d. (Independent and Identically Distributed) and non-i.i.d. (data heterogeneity) settings to simulate real-world scenarios where different users have very different types of data.

Key Insights and Ideas

  • The Power-Weight Link: The most elegant "aha moment" is linking transmit power directly to the model weight ($\rho_k$). Usually, communication and learning are treated as separate layers; here, the communication parameter (power) is the learning parameter (weight).
  • Four Sources of Error: The paper breaks down the learning performance gap into four distinct physical components:
  • Stochastic gradient noise.
  • Data heterogeneity (how different the devices' data are).
  • Downlink noise (errors in receiving the global model).
  • Uplink noise (errors in the aggregated update).
  • Downlink Matters: Many papers ignore the downlink, assuming it's perfect. This work shows that weak downlink channels can "damage" the local model before training even begins, and their optimization accounts for this.

Key Results

The proposed scheme significantly outperformed traditional baselines like "Truncated Channel Inversion" (which often fails to converge in noisy settings).

  • MNIST Performance: On the i.i.d. MNIST task, the proposed scheme reached 92.75% accuracy, nearly matching the 93.09% of "Ideal FedAvg" (which assumes perfect, error-free communication).
  • Robustness to Heterogeneity: In non-i.i.d. settings, where data is unevenly distributed, the proposed scheme maintained high stability, while simpler selection methods (like MSE-thresholding) saw their accuracy drop significantly (e.g., from ~90% down to ~77% in some MNIST cases).
  • CIFAR-10 Success: On the more difficult CIFAR-10 dataset, the proposed scheme achieved 71.52% accuracy, outperforming the MSE-thresholding baseline (66.47%) and proving it can handle complex models like CNNs.

Limitations and Future Work

  • Perfect CSI Assumption: The current design assumes the Parameter Server has perfect Channel State Information (CSI) for all devices. In a real-world high-speed mobile environment, getting perfect CSI is difficult and adds overhead.
  • Static Channels: The analysis assumes channels are "quasi-static" (stable during one round). Future work could explore more rapidly fading environments.
  • Scalability: While the SDR approach is efficient, solving it for thousands of devices simultaneously might still be computationally taxing for a real-time edge server.

Reference: Guo, W., et al. "Joint Device Selection and Power Control for Wireless Federated Learning." arXiv:2205.09306v1, 2022.

Summary updated

Abstract

This paper studies the joint device selection and power control scheme for wireless federated learning (FL), considering both the downlink and uplink communications between the parameter server (PS) and the terminal devices. In each round of model training, the PS first broadcasts the global model to the terminal devices in an analog fashion, and then the terminal devices perform local training and upload the updated model parameters to the PS via over-the-air computation (AirComp). First, we propose an AirComp-based adaptive reweighing scheme for the aggregation of local updated models, where the model aggregation weights are directly determined by the uplink transmit power values of the selected devices and which enables the joint learning and communication optimization simply by the device selection and power control. Furthermore, we provide a convergence analysis for the proposed wireless FL algorithm and the upper bound on the expected optimality gap between the expected and optimal global loss values is derived. With instantaneous channel state information (CSI), we formulate the optimality gap minimization problems under both the individual and sum uplink transmit power constraints, respectively, which are shown to be solved by the semidefinite programming (SDR) technique. Numerical results reveal that our proposed wireless FL algorithm achieves close to the best performance by using the ideal <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FedAvg</i> scheme with error-free model exchange and full device participation.
BibTeX
@article{guo2022joint,
  author = {Wei Guo and Ran Li and Chuan Huang and Xiaoqi Qin and Kaiming Shen and Wei Zhang},
  title = {{Joint Device Selection and Power Control for Wireless Federated Learning}},
  journal = {IEEE J. Sel. Areas Commun.},
  year = {2022},
  url = {https://openreview.net/forum?id=JFkDlWc1E1t},
}

Loading related content...

Figure