Blockchain-Enabled Asynchronous Federated Learning in Edge Computing

Yinghui Liu, Youyang Qu, Chenhao Xu, Zhicheng Hao, Bruce Gu

TL;DR

This paper introduces FedAC, an asynchronous federated learning (FL) approach that uses a blockchain for decentralized model aggregation. By addressing the inefficiencies and vulnerabilities of traditional synchronous FL, FedAC offers improved robustness and efficiency, achieving high accuracy on benchmark datasets.

What Problem Does This Paper Address?

The explosion of edge computing devices generates vast amounts of data, fueling the development of machine learning (ML). However, collecting this data for ML training raises significant privacy concerns. Federated Learning (FL) emerged as a solution, allowing models to be trained on decentralized data without direct data sharing. Traditional FL often relies on synchronous aggregation, where all devices must complete their training rounds before the global model is updated. This approach is inefficient due to varying device capabilities and data sizes, leading to delays. Furthermore, the centralized server used for aggregation is a single point of failure and vulnerable to attacks. This paper tackles these issues by proposing a decentralized, asynchronous FL framework.

Methods

The paper introduces two key components:

  • FedAC (Federated Learning with Asynchronous Convergence): This method addresses the inefficiency of synchronous FL by allowing devices to train and upload their model updates asynchronously. It incorporates a "staleness coefficient" to manage the contribution of delayed updates to the global model, preventing them from halting the entire process. This asynchronous approach improves efficiency by avoiding idle waiting times for faster devices.
  • FedBlock (Blockchain-Enabled Federated Learning): To overcome the single-point-of-failure and security vulnerabilities of a central server, FedBlock replaces it with a blockchain network. The blockchain handles the global model aggregation through a consensus mechanism. This decentralization enhances robustness against attacks and system failures.

The overall framework combines these two to create a robust, asynchronous, and decentralized FL system.

Datasets

  • MNIST: Used for evaluating the proposed models in both horizontal and vertical FL scenarios. This is a standard dataset for image recognition tasks.
  • CIFAR-10: Used for evaluating the performance of asynchronous vs. synchronous modes on a different dataset.

No new benchmarks were introduced; standard datasets were utilized.

Key Insights and Ideas

  • Asynchronous Training is Crucial for Edge: The inherent heterogeneity of edge devices (varying computing power, network connectivity, and data volumes) makes synchronous FL impractical. Asynchronous training, managed by a staleness coefficient, is essential for efficient FL in such environments.
  • Blockchain for Decentralized Trust: Replacing the central server with a blockchain provides a decentralized and tamper-proof mechanism for global model aggregation. This significantly enhances security and resilience against single-point failures and malicious attacks.
  • Trade-offs in Vertical FL: In vertical FL, where each device has unique features, increasing the number of participating devices can sometimes lead to lower accuracy due to data distribution skew. This highlights the importance of considering data characteristics when designing FL systems.
  • Communication Cost as a Bottleneck: The paper identifies communication cost as a major bottleneck for future FL development, suggesting that reducing communication rounds (e.g., by increasing batch size) is a key area for improvement.

Key Results

  • High Accuracy: The proposed FedAC model achieved impressive accuracy rates: 98.96% in horizontal FL and 95.84% in vertical FL on the MNIST dataset.
  • Robustness: By eliminating the central server and enabling asynchronous updates, the system is more resilient to device failures and attacks.
  • Efficiency: Asynchronous convergence avoids delays caused by slower devices, leading to improved overall training efficiency.
  • Comparable Performance: The simulation results show that the proposed framework's performance is comparable to existing synchronous FL methods, while offering enhanced robustness and efficiency.
  • Optimal Blockchain Generation Rate: The study identified an optimal blockchain generation rate (β) that minimizes convergence latency, demonstrating the feasibility of the blockchain integration.

Limitations and Future Work

  • Non-IID Data: While the paper's FedAC model can work with both horizontal and vertical FL, it is not specifically designed to cope with Non-IID (Non-Independent and Identically Distributed) issues. The authors acknowledge this as a significant challenge for future work, especially in real-world heterogeneous environments.
  • Larger Scale Scenarios: The authors plan to focus on scenarios with a larger number of distributed devices in future work, recognizing that more participants introduce more undiscovered factors that can influence training.
  • Privacy Enhancements: Future research will also address privacy issues in blockchain-enabled FL by exploring differential privacy and other advanced techniques.

Summary updated

Abstract

The fast proliferation of edge computing devices brings an increasing growth of data, which directly promotes machine learning (ML) technology development. However, privacy issues during data collection for ML tasks raise extensive concerns. To solve this issue, synchronous federated learning (FL) is proposed, which enables the central servers and end devices to maintain the same ML models by only exchanging model parameters. However, the diversity of computing power and data sizes leads to a significant difference in local training data consumption, and thereby causes the inefficiency of FL. Besides, the centralized processing of FL is vulnerable to single-point failure and poisoning attacks. Motivated by this, we propose an innovative method, federated learning with asynchronous convergence (FedAC) considering a staleness coefficient, while using a blockchain network instead of the classic central server to aggregate the global model. It avoids real-world issues such as interruption by abnormal local device training failure, dedicated attacks, etc. By comparing with the baseline models, we implement the proposed method on a real-world dataset, MNIST, and achieve accuracy rates of 98.96% and 95.84% in both horizontal and vertical FL modes, respectively. Extensive evaluation results show that FedAC outperforms most existing models.
BibTeX
@inproceedings{liu2021blockchain,
  author = {Yinghui Liu and Youyang Qu and Chenhao Xu and Zhicheng Hao and Bruce Gu},
  title = {{Blockchain-Enabled Asynchronous Federated Learning in Edge Computing}},
  booktitle = {Sensors},
  year = {2021},
  url = {https://openreview.net/forum?id=fOYNrQ7SBT},
}
Figure