Abstract
This paper introduces an autoencoder-based approach to design digital signal constellations specifically for Over-the-Air Computation (AirComp) in Federated Learning (FL). By jointly optimizing transmitters and receivers, the authors overcome the "summation ambiguity" problem inherent in digital modulation, allowing a parameter server to accurately decode the sum of model updates from multiple devices simultaneously.
Introduction: The Challenge of Digital AirComp
In Federated Learning (FL), edge devices train local models and transmit updates to a central server for aggregation. The communication uplink often becomes a bottleneck. AirComp addresses this by allowing simultaneous transmissions that naturally sum in the wireless channel. While well-studied for analog transmission, digital AirComp faces "summation ambiguity."
As shown in the figure above, standard digital constellations like QPSK can lead to scenarios where the receiver cannot distinguish between different sums (e.g., "00+11" vs "01+10"). This ambiguity grows exponentially with the number of bits or clients, making handcrafted designs impractical for high-order modulations.
Proposed Method: Autoencoder-Based Design
The authors model the end-to-end communication system as an Autoencoder (AE) to learn optimal constellation geometries that are naturally optimized for summation.
The architecture consists of $n$ encoders (edge devices) and one shared decoder (parameter server). The process involves:
- Digital Pipeline: Model updates undergo quantization (scaling, stochastic rounding, and bit-limiting) before being mapped to complex constellation points by the encoder.
- Joint Optimization: The AE is trained end-to-end to minimize the error between the intended sum of messages and the decoded sum, accounting for channel fading and additive noise.
- Streamlining Categorization: To manage complexity in high-order modulations (e.g., 121 possible sums for 8 clients), the authors use "coarse-grained classification," grouping numerous possible sums into a smaller set of categories. This makes the system scalable and more robust to noise.
Experimental Evaluation
The researchers evaluated their method using the CIFAR-10 dataset under both IID (Independent and Identically Distributed) and non-IID data distributions to simulate realistic federated learning environments.
Key Insights
- Learning the Constellation: Instead of manual design, the neural network "learns" a constellation geometry that avoids the overlaps that plague standard digital designs.
- The "Streamlining" Trick: High FL accuracy can be maintained without decoding the exact sum. Grouping similar sum values into categories reduces the complexity of the output layer (softmax) without significantly hurting final model performance.
- Robustness to Noise: The AE-based design is remarkably resilient. Even at low Signal-to-Noise Ratios (SNR), the learned constellations maintain enough distance between "sum-categories" to allow the FL process to converge.
Results
- Near-Perfect Performance: In high-SNR environments (15 dB), the proposed
AE_optmethod achieved performance virtually identical to an ideal scenario with zero transmission error. - Low SNR Resilience: Even at a challenging 0 dB SNR, the system showed only a minor drop in accuracy compared to the ideal case.
- Scalability: The streamlining approach successfully enabled 4-bit modulation for 8 clients, a scenario that would typically be too complex for standard digital AirComp designs.
- Non-IID Success: The method remained effective even when clients had very different local datasets, though adjusting "quantization gain" in later training rounds helps mitigate accuracy drops as the model converges.
Conclusion and Future Work
While the autoencoder-based approach significantly improves digital AirComp for FL, challenges remain. Future work will focus on refining the system's sensitivity to quantization as model updates become smaller near convergence. Additionally, the authors aim to optimize for dynamic client participation and even higher-order modulations beyond 4-bit.
Summary updated