Relay-Assisted Cooperative Federated Learning

Zehong Lin, Hang Liu, Ying-Jun Angela Zhang

Abstract

Federated learning (FL) has recently emerged as a promising technology to enable artificial intelligence (AI) at the network edge, where distributed mobile devices collaboratively train a shared AI model under the coordination of an edge server. To significantly improve the communication efficiency of FL, over-the-air computation allows a large number of mobile devices to concurrently upload their local models by exploiting the superposition property of wireless multi-access channels. Due to wireless channel fading, the model aggregation error at the edge server is dominated by the weakest channel among all devices, causing severe straggler issues. In this paper, we propose a relay-assisted cooperative FL scheme to effectively address the straggler issue. In particular, we deploy multiple half-duplex relays to cooperatively assist the devices in uploading the local model updates to the edge server. The nature of the over-the-air computation poses system objectives and constraints that are distinct from those in traditional relay communication systems. Moreover, the strong coupling between the design variables renders the optimization of such a system challenging. To tackle the issue, we propose an alternating-optimization-based algorithm to optimize the transceiver and relay operation with low complexity. Then, we analyze the model aggregation error in a single-relay case and show that our relay-assisted scheme achieves a smaller error than the one without relays provided that the relay transmit power and the relay channel gains are sufficiently large. The analysis provides critical insights on relay deployment in the implementation of cooperative FL. Extensive numerical results show that our design achieves faster convergence compared with state-of-the-art schemes.

Introduction

Federated Learning (FL) allows mobile devices to train a shared AI model without sharing their raw data, but the communication overhead is a massive bottleneck. To speed things up, researchers use Over-the-Air Computation (AirComp), which exploits the physics of wireless channels to "sum" local model updates as they are transmitted simultaneously.

However, AirComp has a major Achilles' heel: the straggler issue. To ensure the signals align correctly at the edge server (the Access Point or AP), devices with strong channels must "throttle" their power to match the device with the weakest channel. This results in a low signal-to-noise ratio (SNR) for the entire aggregate model, leading to high errors. Previous solutions simply kicked the stragglers out of the training round, but this wastes valuable data and slows down the model's ability to learn.

System Architecture

The proposed framework introduces a cooperative relaying mechanism to support devices with poor channel conditions. By deploying half-duplex relays, the system can amplify and forward signals to the AP, effectively boosting the weakest links in the network.

Relay-assisted cooperative FL system model with K devices and N relays.

As shown in the system model, the process is divided into two distinct phases. In the first phase, devices broadcast their local model updates to both the AP and the relays. In the second phase, the relays and the devices transmit simultaneously to the AP, providing diversity and signal reinforcement.

The Two-Phase Relaying Scheme

The core of this approach is a two-phase Amplify-and-Forward (AF) relaying scheme designed to keep stragglers active in the training process:

  1. Phase 1 (Broadcast): Devices transmit their model updates. Both the AP and a set of intermediate relays receive these signals.
  2. Phase 2 (Cooperative Relay & Retransmission): The relays amplify what they heard and forward it to the AP. Crucially, the devices also re-transmit their signals in this phase. The AP then combines the signals from both phases to form a much cleaner estimate of the global model update.
Detailed signal flow for the two-phase transmission and aggregation process.

Because the resulting optimization problem (minimizing Mean Square Error or MSE) is non-convex and "strongly coupled," the authors developed an Alternating Optimization (AO) algorithm. This algorithm iteratively tweaks the transmit power of the devices, the relaying scalars, and the receiver's de-noising factors until it reaches a stable, high-performance configuration.

Key Insights and Theoretical Analysis

  • Cooperative Diversity: Unlike previous relaying attempts where devices go silent while relays talk, this paper keeps devices active in both phases. This "cooperation" between devices and relays provides a diversity gain that makes the system much more robust.
  • Relay Deployment Guidelines: A standout contribution is Theorem 4.1, which mathematically defines the conditions under which a relay actually helps. If the relay-to-AP link is too noisy, the relay just ends up amplifying noise, which can be worse than having no relay at all. This provides a practical guideline for where to physically place relays in a network.
  • Robustness to Placement: The proposed design is remarkably insensitive to exactly where the relay is located, provided it meets the basic SNR conditions. This is a significant advantage over "relay-only" baselines that fail if the relay isn't perfectly positioned.

Experimental Results

The researchers evaluated their framework using the Fashion-MNIST dataset and simulated wireless channels using a Rayleigh fading model. Key results include:

  • Lower Error: In high-noise environments (-70 dBm), the proposed scheme achieved a Normalized Mean Square Error (NMSE) of -6.29 dB, while the standard FL approach without relays suffered a massive error of +4.79 dB.
  • Faster Convergence: The relay-assisted model reached high test accuracy (approx. 80%) in significantly fewer transmission blocks compared to non-cooperative baselines.
  • Robustness to Non-I.I.D. Data: In scenarios where different devices have very different types of data, the relay-assisted approach maintained high accuracy where other methods saw performance collapses.

Conclusion

This relay-assisted framework for AirComp-based FL effectively solves the straggler problem by utilizing cooperative diversity. By reducing model aggregation errors and accelerating training convergence, it enables more efficient AI training at the network edge. Future work will explore full-duplex relays and multi-cell coordination to further reduce latency and interference.

Summary updated

Abstract

Federated learning (FL) has recently emerged as a promising technology to enable artificial intelligence (AI) at the network edge, where distributed mobile devices collaboratively train a shared AI model under the coordination of an edge server. To significantly improve the communication efficiency of FL, over-the-air computation allows a large number of mobile devices to concurrently upload their local models by exploiting the superposition property of wireless multi-access channels. Due to wireless channel fading, the model aggregation error at the edge server is dominated by the weakest channel among all devices, causing severe straggler issues. In this paper, we propose a relay-assisted cooperative FL scheme to effectively address the straggler issue. In particular, we deploy multiple half-duplex relays to cooperatively assist the devices in uploading the local model updates to the edge server. The nature of the over-the-air computation poses system objectives and constraints that are distinct from those in traditional relay communication systems. Moreover, the strong coupling between the design variables renders the optimization of such a system challenging. To tackle the issue, we propose an alternating-optimization-based algorithm to optimize the transceiver and relay operation with low complexity. Then, we analyze the model aggregation error in a single-relay case and show that our relay-assisted scheme achieves a smaller error than the one without relays provided that the relay transmit power and the relay channel gains are sufficiently large. The analysis provides critical insights on relay deployment in the implementation of cooperative FL. Extensive numerical results show that our design achieves faster convergence compared with state-of-the-art schemes.
BibTeX
@article{lin2022relay,
  author = {Zehong Lin and Hang Liu and Ying-Jun Angela Zhang},
  title = {{Relay-Assisted Cooperative Federated Learning}},
  journal = {IEEE Transactions on Wireless Communications},
  year = {2022},
  url = {https://openreview.net/forum?id=UbTEqCTb3O},
}
Figure