Abstract
Decentralized learning offers privacy and scalability benefits by removing the need for a central server. However, real-world wireless edge networks introduce challenges such as intermittent connectivity and heterogeneous device performance. This paper proposes a robust asynchronous Decentralized Stochastic Gradient Descent (DSGD) algorithm that leverages stale gradient reuse and time-varying mixing to overcome these hurdles. Experimental results demonstrate that the proposed method achieves faster convergence and higher accuracy compared to traditional synchronous approaches in unreliable network environments.
Introduction
Decentralized learning enables devices to collaboratively train a shared model while keeping data local. Despite its potential, practical deployment in wireless edge networks is hindered by two primary factors: communication failures due to fading or blockages, and computation stragglers caused by hardware variability. Traditional synchronous algorithms often suffer from the "slowest node" bottleneck, while "delay barrier" methods discard valuable data from slower participants. This work addresses these issues by introducing an asynchronous framework that maintains progress using available information, even if it is outdated.
Methodology
The proposed Asynchronous Decentralized SGD algorithm incorporates several key innovations to handle the "messy" reality of wireless networks:
Stale Gradient Reuse and Heterogeneous Learning Rates
Instead of discarding updates from slow nodes (stragglers), the algorithm allows nodes to apply gradients computed in previous iterations. To ensure convergence despite varying update frequencies across nodes, the authors mathematically tune local learning rates, ensuring the model moves toward the global optimum.
Time-Varying Mixing and AirComp
The network topology is treated as dynamic. Using a consensus strategy based on Metropolis-Hastings weighting, nodes average their models with reachable neighbors at any given time. To enhance communication efficiency, Over-the-Air Computation (AirComp) is employed, where analog signals physically overlap to perform weighted sums, reducing latency.
Experimental Evaluation
The approach was evaluated using the Fashion-MNIST dataset on a CNN model trained by 15 simulated wireless edge devices. The environment modeled Rayleigh fading channels and exponential computation time distributions.
Key Results
The theoretical analysis provides a non-asymptotic convergence guarantee of $O(T^{-1/4})$. Empirically, the asynchronous approach proves superior in constrained environments. When a strict time limit is imposed ($T_{max} = \frac{4}{5} \mathbb{E}[T_{comp}]$), Asynchronous DSGD significantly outperforms Synchronous DSGD with a delay barrier, achieving 5-10% higher accuracy within the same wall-clock time.
Conclusion
This paper demonstrates that "stale" information is often more valuable than "discarded" information in decentralized learning. By embracing asynchrony and adapting to dynamic network topologies, the proposed DSGD algorithm offers a resilient solution for wireless edge intelligence. Future work may explore sophisticated weighting for stale gradients and advanced communication compression techniques to further improve scalability.
Summary updated