Abstract
Federated Learning (FL) is a promising paradigm for training machine learning models on decentralized edge devices while preserving data privacy. However, the practical deployment of FL is often hindered by the "bandwidth gap" between high-dimensional model updates and limited wireless channel capacities. This paper presents Band-limited Coordinated Descent (BLCD), an approach that treats the wireless channel as an integral part of the computation. By combining over-the-air (OTA) computing, coordinate sparsification, and joint power-learning rate optimization, BLCD enables efficient model convergence in noisy, band-limited environments.
Introduction
Standard Federated Learning assumes lossless communication between edge devices and a central server. In reality, edge devices like IoT sensors and smartphones operate over narrow, noisy, and power-constrained wireless channels. High-dimensional gradient updates (often millions of parameters) must be transmitted over limited subcarriers, leading to significant communication overhead and potential training instability due to channel fading and noise.
Existing solutions typically decouple communication (source/channel coding) from the learning process. In contrast, the proposed Band-limited Coordinated Descent (BLCD) framework adopts a holistic approach, designing the communication strategy specifically to support the requirements of Stochastic Gradient Descent (SGD).
System Architecture and BLCD Framework
The BLCD algorithm leverages several key techniques to bridge the bandwidth gap:
- Over-the-Air (OTA) Computing: Instead of orthogonal resource allocation (e.g., TDMA/FDMA), all devices transmit simultaneously on the same frequency. The wireless medium naturally performs a weighted summation of signals via superposition, allowing the server to receive the aggregate gradient directly.
- Coordinate Sparsification: To accommodate limited bandwidth, the receiver selects a common subset of $K$ coordinates (out of $d$ total dimensions) for all devices to transmit in a given round.
- Error Feedback: Devices maintain a local memory of gradients that were not transmitted due to sparsification, adding them to subsequent updates to ensure no information is permanently lost.
Communication Protocol
The BLCD process operates in discrete rounds. Each round involves local computation at the devices, a coordination phase where the server determines and broadcasts the active coordinate subset $I_t$, and a transmission phase where devices send their sparsified updates.
The authors formulate a joint optimization problem to determine the optimal power allocation for each device and the global learning rate. They propose three schemes:
- Scheme 1 (Centralized): A bi-convex optimization benchmark.
- Scheme 2 (Distributed Zero-Bias): A practical scheme that eliminates communication bias to ensure stable convergence.
- Scheme 3 (User-Centric): A simplified "water-filling" approach based on local channel conditions.
Experimental Results
The BLCD framework was evaluated using the MNIST dataset across three architectures: a single-layer NN, a 5-layer DNN, and an 18-layer ResNet (11M+ parameters). Simulations were conducted in a Rayleigh fading environment with 8-10 workers.
Key findings include:
- Convergence under Noise: Scheme 1 achieved performance nearly identical to error-free scenarios, even in noisy wireless conditions.
- Bias Mitigation: Gradient bias, rather than just Mean Squared Error (MSE), was identified as the primary bottleneck for convergence. The "Zero-Bias" scheme significantly outperformed naive power allocation.
- Scalability: The approach successfully trained the ResNet-18 model, demonstrating its viability for large-scale edge intelligence.
Conclusion
BLCD provides a robust theoretical and practical foundation for communication-efficient AI. By exploiting the physical properties of the wireless channel and integrating them into the optimization loop, the framework achieves high performance despite severe bandwidth and power constraints. Future work will address challenges in time/phase synchronization and more advanced coordinate selection strategies like Top-K sparsification.
Summary updated