Abstract
Adversarial Federated Learning (AFL) is essential for securing decentralized systems against malicious perturbations, yet it is often impractical for resource-constrained edge devices due to high communication costs and computational demands. We present PM-AFL++, a framework that leverages knowledge distillation from a fixed, pre-trained teacher model and a novel global alignment strategy. PM-AFL++ achieves superior adversarial robustness and clean accuracy while reducing communication overhead by up to 73x compared to traditional AFL methods.
Introduction
Federated Learning (FL) enables multiple devices to collaboratively train a shared model without exchanging raw data, preserving privacy. However, FL systems face two critical challenges: adversarial vulnerability, where small input perturbations can deceive models, and edge constraints, where limited bandwidth and compute power make standard Adversarial Training (AT) unfeasible. Furthermore, the non-IID (not identically distributed) nature of edge data often leads to "client drift," where local models diverge from the global objective. PM-AFL++ addresses these issues by using a pre-trained "teacher" model as a local anchor, guiding efficient and robust student model training.
Methodology: PM-AFL++
Instead of training a robust model from scratch across the network, PM-AFL++ utilizes a high-performance, pre-trained teacher model (e.g., WideResNet) to guide smaller student models on each device. The framework integrates three core loss components to balance accuracy, robustness, and global consistency:
- Vanilla Mixture Knowledge Distillation ($\mathcal{L}_{VKD}$): Student models learn to mimic teacher predictions on both natural and "mixed" images (using the Mixup technique) to maintain high clean accuracy.
- Adversarial Mixture Knowledge Distillation ($\mathcal{L}_{AKD}$): To build robustness, students are trained to match the teacher's outputs for clean images when presented with adversarial versions. This distills the teacher's stability into the student.
- Global Alignment ($\mathcal{L}_{ALG}$): To mitigate non-IID data effects, this term aligns the local student's adversarial features with the global model's clean features, acting as a regularizer against client drift.
Crucially, the teacher model remains local to each client and is never transmitted, ensuring that communication costs are dictated solely by the compact student model parameters.
Experimental Results
We evaluated PM-AFL++ on MNIST, CIFAR-10, and CIFAR-100 benchmarks under highly non-IID conditions. The framework consistently outperformed state-of-the-art baselines like FedPGD and DBFAT in both efficiency and defense capability.
Communication Efficiency
PM-AFL++ achieves massive reductions in communication overhead. By distilling knowledge into a tiny student model, it reduced the number of parameters communicated per round by 73x on MNIST and 36x on CIFAR-10 compared to standard AFL methods. This makes robust federated learning viable for low-bandwidth IoT environments.
Robustness and Accuracy
Under the rigorous AutoAttack (AA) evaluation, PM-AFL++ demonstrated superior resilience. On CIFAR-10, it achieved a robust accuracy of 20.22%, significantly higher than FedPGD (17.22%) and FedAvg (0.00%). Unlike many defense methods that suffer from a "robustness-accuracy trade-off," PM-AFL++ also maintained higher clean accuracy (47.88% on CIFAR-10 vs. 28.82% for FedPGD).
Key Insights
- The Teacher as an Anchor: A fixed teacher provides a consistent reference point, preventing the "blind leading the blind" scenario common in early-stage AFL.
- Feature-Level Regularization: The $\mathcal{L}_{ALG}$ term ensures that local models "think" about adversarial images in a way that aligns with the global model's understanding of clean data, effectively handling data heterogeneity.
- Asymmetric Distillation: Using a powerful teacher to train a tiny student allows for high-performance models that are small enough for edge deployment.
Conclusion and Future Work
PM-AFL++ represents a significant step toward making Adversarial Federated Learning practical for the edge. While the framework's performance is currently linked to the availability of a suitable pre-trained teacher, its ability to drastically reduce communication costs while improving robustness is a major advancement. Future research will focus on handling heterogeneous client architectures and further optimizing the accuracy-robustness balance.
Summary updated