Paving the Way for Distributed ArtificialIntelligence over the Air

Guoqing Ma, Shuping Dang, Chuanting Zhang, Basem Shihada

Abstract: Distributed Artificial Intelligence (DAI) is regarded as one of the most promising techniques to provide intelligent services under strict privacy protection regulations for multiple clients. By applying DAI, training on raw data is carried out locally, while the trained outputs, e.g., model parameters, from multiple local clients, are sent back to a central server for aggregation. Recently, for achieving better practicality, DAI is studied in conjunction with wireless communication networks, incorporating various random effects brought by wireless channels. However, because of the complex and case-dependent nature of wireless channels, a generic simulator for applying DAI in wireless communication networks is still lacking. To accelerate the development of DAI applied in wireless communication networks, we propose a generic system design in this paper as well as an associated simulator that can be set according to wireless channels and system-level configurations. Details of the system design and analysis of the impacts of wireless environments are provided to facilitate further implementations and updates. We employ a series of experiments to verify the effectiveness and efficiency of the proposed system design and reveal its superior scalability.

Introduction

Distributed Artificial Intelligence (DAI), particularly Federated Learning (FL), is a cornerstone for privacy-preserving machine learning. By keeping raw data on local devices and only sharing model updates, DAI meets strict privacy regulations. However, most existing research operates under the assumption of "perfect" network conditions. In reality, wireless channels are plagued by path loss, fading, and packet errors. To address this, we introduce AirDAI, an open-source framework and simulator designed to evaluate DAI performance in realistic, messy wireless environments.

System Design and Methodology

AirDAI virtualizes the DAI process through two primary lenses: spatial and temporal.

Spatial Perspective

The system decomposes tasks into client-side local training and server-side global aggregation. This architecture is modeled using a hierarchical tree structure where nodes represent servers, access points (APs), and clients, each with specific wireless attributes like mobility and channel type.

The architecture of AirDAI involves local training on diverse client devices and global aggregation at a central DDL server via wireless uplink and downlink.

Temporal Perspective

The training process is organized into discrete "rounds." Each round consists of three phases: broadcast, local training, and aggregation. This temporal modeling allows the simulator to account for real-world issues like device failures, network timeouts, and varying compute capabilities.

AirDAI models the temporal flow of a training round, accounting for processing time, device sleeping states, and potential network failures.

Key Insights and Experimental Results

The framework was validated using the MNIST image classification dataset and the Telecom Italy Call Detail Record (CDR) dataset for traffic prediction.

The "Wireless Tax" on Convergence

Theoretical analysis and simulations reveal that wireless effects act as a "tax" on convergence. Packet losses and partial client participation effectively modify the system's learning rate, potentially slowing down or destabilizing the training process.

Task-Dependent Bottlenecks

Experiments showed that bottlenecks are task-specific. For image classification (MNIST), the primary delay is local computation. Conversely, for traffic prediction (CDR), communication overhead dominates the total time per round.

Robustness to "Naughty" Clients

The authors introduced "naughty" clients that inject additive or multiplicative noise into the training process. They found that different tasks have vastly different tolerance levels; for instance, MNIST classification is much more sensitive to additive noise than traffic prediction.

Scalability and Efficiency

AirDAI leverages the Message Passing Interface (MPI) to distribute simulations across multiple CPU cores. Testing showed near-linear scalability; for a 32-cell simulation, increasing from 1 to 16 cores reduced the wall-clock time per round by approximately 90%.

Conclusion

AirDAI provides a robust, scalable platform for bridging the gap between machine learning and wireless communications. By accounting for the stochastic nature of wireless channels, it enables researchers to design DAI algorithms that are truly ready for deployment in 6G networks and IoT ecosystems.

Code available at: https://github.com/KAUST-Netlab/AirDAI

Summary updated

Abstract

Distributed Artificial Intelligence (DAI) is regarded as one of the most promising techniques to provide intelligent services under strict privacy protection regulations for multiple clients. By applying DAI, training on raw data is carried out locally, while the trained outputs, e.g., model parameters, from multiple local clients, are sent back to a central server for aggregation. Recently, for achieving better practicality, DAI is studied in conjunction with wireless communication networks, incorporating various random effects brought by wireless channels. However, because of the complex and case-dependent nature of wireless channels, a generic simulator for applying DAI in wireless communication networks is still lacking. To accelerate the development of DAI applied in wireless communication networks, we propose a generic system design in this paper as well as an associated simulator that can be set according to wireless channels and system-level configurations. Details of the system design and analysis of the impacts of wireless environments are provided to facilitate further implementations and updates. We employ a series of experiments to verify the effectiveness and efficiency of the proposed system design and reveal its superior scalability.
BibTeX
@article{ma2022paving,
  author = {Guoqing Ma and Shuping Dang and Chuanting Zhang and Basem Shihada},
  title = {{Paving the Way for Distributed ArtificialIntelligence over the Air}},
  journal = {IEEE Open J. Commun. Soc.},
  year = {2022},
  url = {https://openreview.net/forum?id=VmKTlD5PcB},
}
Figure