An Autoencoder-Based Constellation Design for AirComp in Wireless Federated Learning

Yujia Mu, Xizixiang Wei, Cong Shen

Abstract

This paper introduces an autoencoder-based approach to design digital signal constellations specifically for Over-the-Air Computation (AirComp) in Federated Learning (FL). By jointly optimizing transmitters and receivers, the authors overcome the "summation ambiguity" problem inherent in digital modulation, allowing a parameter server to accurately decode the sum of model updates from multiple devices simultaneously.

Introduction: The Challenge of Digital AirComp

In Federated Learning (FL), edge devices train local models and transmit updates to a central server for aggregation. The communication uplink often becomes a bottleneck. AirComp addresses this by allowing simultaneous transmissions that naturally sum in the wireless channel. While well-studied for analog transmission, digital AirComp faces "summation ambiguity."

Illustration of summation ambiguity where different signal pairs result in overlapping sum points.

As shown in the figure above, standard digital constellations like QPSK can lead to scenarios where the receiver cannot distinguish between different sums (e.g., "00+11" vs "01+10"). This ambiguity grows exponentially with the number of bits or clients, making handcrafted designs impractical for high-order modulations.

Proposed Method: Autoencoder-Based Design

The authors model the end-to-end communication system as an Autoencoder (AE) to learn optimal constellation geometries that are naturally optimized for summation.

Detailed architecture of the proposed Autoencoder-based AirComp system featuring n encoders and a shared decoder.

The architecture consists of $n$ encoders (edge devices) and one shared decoder (parameter server). The process involves:

  1. Digital Pipeline: Model updates undergo quantization (scaling, stochastic rounding, and bit-limiting) before being mapped to complex constellation points by the encoder.
  2. Joint Optimization: The AE is trained end-to-end to minimize the error between the intended sum of messages and the decoded sum, accounting for channel fading and additive noise.
  3. Streamlining Categorization: To manage complexity in high-order modulations (e.g., 121 possible sums for 8 clients), the authors use "coarse-grained classification," grouping numerous possible sums into a smaller set of categories. This makes the system scalable and more robust to noise.

Experimental Evaluation

The researchers evaluated their method using the CIFAR-10 dataset under both IID (Independent and Identically Distributed) and non-IID data distributions to simulate realistic federated learning environments.

Key Insights

  • Learning the Constellation: Instead of manual design, the neural network "learns" a constellation geometry that avoids the overlaps that plague standard digital designs.
  • The "Streamlining" Trick: High FL accuracy can be maintained without decoding the exact sum. Grouping similar sum values into categories reduces the complexity of the output layer (softmax) without significantly hurting final model performance.
  • Robustness to Noise: The AE-based design is remarkably resilient. Even at low Signal-to-Noise Ratios (SNR), the learned constellations maintain enough distance between "sum-categories" to allow the FL process to converge.

Results

  • Near-Perfect Performance: In high-SNR environments (15 dB), the proposed AE_opt method achieved performance virtually identical to an ideal scenario with zero transmission error.
  • Low SNR Resilience: Even at a challenging 0 dB SNR, the system showed only a minor drop in accuracy compared to the ideal case.
  • Scalability: The streamlining approach successfully enabled 4-bit modulation for 8 clients, a scenario that would typically be too complex for standard digital AirComp designs.
  • Non-IID Success: The method remained effective even when clients had very different local datasets, though adjusting "quantization gain" in later training rounds helps mitigate accuracy drops as the model converges.

Conclusion and Future Work

While the autoencoder-based approach significantly improves digital AirComp for FL, challenges remain. Future work will focus on refining the system's sensitivity to quantization as model updates become smaller near convergence. Additionally, the authors aim to optimize for dynamic client participation and even higher-order modulations beyond 4-bit.

Summary updated

BibTeX
@inproceedings{mu2024autoencoder,
  author = {Yujia Mu and Xizixiang Wei and Cong Shen},
  title = {{An Autoencoder-Based Constellation Design for AirComp in Wireless Federated Learning}},
  booktitle = {ICC},
  year = {2024},
  url = {https://openreview.net/forum?id=TCRGz9y7Jn},
}
Figure