Towards Communication-Efficient Adversarial Federated Learning for Robust Edge Intelligence

Yu Qiao, Huy Q. Le, Apurba Adhikary, Choong Seon Hong, Han, Zhu, Hong, Choong Seon

Abstract

Federated learning (FL) enables decentralized training on edge networks without exposing raw data. However, FL models remain susceptible to adversarial attacks and performance degradation in non-IID data settings. This paper proposes a pre-trained model-guided adversarial federated learning (PM-AFL++) framework. By leveraging knowledge from a local pre-trained teacher model for both clean and adversarial images, we propose a mixture knowledge distillation strategy that balances accuracy and robustness. Our framework significantly outperforms existing methods while slashing communication costs by up to 73x.

Introduction

Federated Learning (FL) allows multiple edge devices to collaboratively train a global model. However, it faces two major hurdles: Adversarial Vulnerability, where models are fooled by imperceptible perturbations, and Non-IID Data, which degrades generalization. Standard Adversarial Training (AT) is computationally exhausting for edge devices and requires massive communication overhead. PM-AFL++ addresses this by using a local robust teacher to guide a smaller student model, avoiding the need to transmit massive robust models over the network.

Methodology

PM-AFL++ integrates three core components to achieve robustness and efficiency:

  • Vanilla Mixture Knowledge Distillation (VKD): The student model learns from the teacher using clean and "mixed" images (interpolated pairs) to maintain high clean accuracy.
  • Adversarial Mixture Knowledge Distillation (AKD): The student processes adversarial examples and aligns its predictions with the teacher's clean predictions to build robustness.
  • Global Alignment (ALG): A consistency regularization term forces local adversarial representations to align with the global model's clean representations, preventing overfitting to skewed local data distributions.
The PM-AFL framework architecture, illustrating the interaction between local models, the global model, and the three distillation/regularization loss terms.

Experimental Results

We evaluated the framework on MNIST, CIFAR-10, and CIFAR-100 using a Dirichlet distribution ($\alpha = 0.1$) to simulate high data heterogeneity.

Comprehensive performance comparison of PM-AFL++ against state-of-the-art AFL methods across MNIST, CIFAR-10, and CIFAR-100 datasets.

Key Findings

  • Superior Accuracy & Robustness: On CIFAR-10, PM-AFL++ achieved 47.88% clean accuracy and 24.03% average robust accuracy, nearly doubling the clean accuracy of the standard FedPGD baseline (28.82%).
  • Massive Communication Reductions: By enabling the use of smaller student models, communication parameters per round dropped from 11.69 million to just 320,000 on CIFAR-10 (a 36x reduction) and achieved a 73x reduction on MNIST.
  • Efficiency: PM-AFL++ required 1.3x to 1.6x fewer communication rounds to converge compared to baselines.
  • Model Capacity: Surprisingly, scaling up student model size (e.g., to WideResNet-34-10) did not always improve performance due to optimization complexities in the AFL landscape, suggesting that moderately sized students strike the best balance.

Conclusion and Future Work

PM-AFL++ demonstrates that knowledge distillation from a robust teacher is a highly effective alternative to traditional adversarial training in federated settings. It successfully mitigates the trade-off between robustness and communication efficiency. Future research will focus on reducing the local computational burden of hosting teacher models and generating adversarial examples on extremely resource-constrained IoT devices.

Summary updated

Abstract

Federated learning (FL) has gained significant attention for enabling decentralized training on edge networks without exposing raw data. However, FL models remain susceptible to adversarial attacks and performance degradation in non-IID data settings, thus posing challenges to both robustness and accuracy. This paper aims to achieve communication-efficient adversarial federated learning (AFL) by leveraging a pre-trained model to enhance both robustness and accuracy under adversarial attacks and non-IID challenges in AFL. By leveraging the knowledge from a pre-trained model for both clean and adversarial images, we propose a pre-trained model-guided adversarial federated learning (PM-AFL) framework. This framework integrates vanilla and adversarial mixture knowledge distillation to effectively balance accuracy and robustness while promoting local models to learn from diverse data. Specifically, for clean accuracy, we adopt a dual distillation strategy where the class probabilities of randomly paired images, and their blended versions are aligned between the teacher model and the local models. For adversarial robustness, we employ a similar distillation approach but replace clean samples on the local side with adversarial examples. Moreover, by considering the bias between local and global models, we also incorporate a consistency regularization term to ensure that local adversarial predictions stay aligned with their corresponding global clean ones. These strategies collectively enable local models to absorb diverse knowledge from the teacher model while maintaining close alignment with the global model, thereby mitigating overfitting to local optima and enhancing the generalization of the global model. Experiments demonstrate that the PM-AFL-based framework not only significantly outperforms other methods but also maintains communication efficiency.
BibTeX
@misc{qiao2025towards,
  author = {Yu Qiao and Huy Q. Le and Apurba Adhikary and Choong Seon Hong and Han, Zhu and Hong, Choong Seon},
  title = {{Towards Communication-Efficient Adversarial Federated Learning for Robust Edge Intelligence}},
  year = {2025},
  eprint = {2501.15257},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.15257},
}
Figure