Abstract Adversarial training (AT) is a prominent technique employed by deep learning models to defend against adversarial attacks. However, existing AT-based methods suffer from high computational costs, low generalization to unseen attacks, and a trade-off between robustness and clean-sample classification accuracy (CCA). We propose the Meta Adversarial Defense (MAD) benchmark, comprising two large-scale datasets (MAD-M and MAD-C) generated using 30 types of attacks, and an evaluation protocol. We introduce a meta-learning based adversarial training (Meta-AT) algorithm as a baseline. Meta-AT treats defense as a few-shot learning problem, enabling high robustness to unseen attacks with minimal adaptation time. Experimental results show Meta-AT outperforms state-of-the-art methods while maintaining high CCA, addressing the core limitations of traditional AT.
Introduction
Adversarial Training (AT) remains the primary defense against malicious perturbations. Despite its effectiveness, it faces three critical challenges: high computational overhead during training, poor generalization to attacks not seen during training, and a significant drop in accuracy on clean data. The MAD benchmark shifts the focus from building static defenses to developing models capable of rapid adaptation.
Methods: Meta-Adversarial Training (Meta-AT)
The Meta-AT algorithm redefines adversarial defense as a meta-learning task. Instead of training on a static set of adversarial examples, the model learns to adapt its parameters quickly when encountering new attack patterns.
Task-Based Training
The process follows an "N-way, K-shot" framework common in meta-learning. During training, the model is exposed to a diverse library of attacks and learns to update its parameters so that it can adapt to any of them with just a few examples.
Dual Updates
As shown in the workflow below, the algorithm performs "inner" updates to specialize for a specific attack and "outer" updates to optimize the base model's generalizability. This structure allows the model to bridge the gap between different attack domains.
Evaluation Metric
The authors introduce the Equilibrium Defense Success Rate (EDSR). Standard metrics only look at defense success; EDSR factors in Operating Time (OT). A defense that takes hours to adapt is less valuable than one that takes seconds. The formula $EDSR = DSR \times e^{-OT}$ penalizes slow adaptation.
The MAD Benchmark Datasets
To support this research, the authors developed two comprehensive datasets:
- MAD-M: Based on MNIST (handwritten digits), featuring 22 different types of attacks.
- MAD-C: Based on CIFAR-10 (common objects), featuring 29 different types of attacks.
- The Attack Library: Includes 30 mainstream attacks, such as PGD, DeepFool, FGSM, and black-box attacks like Square and Zoo-Attack. This variety ensures models are tested against a wide spectrum of adversarial strategies.
Key Results
Meta-AT demonstrates significant improvements across all metrics:
- Superior Robustness: Meta-AT achieved a maximum EDSR of 99.77% on MAD-M and 92.98% on MAD-C, significantly outperforming other state-of-the-art (SOTA) methods like ATTA and YOPO.
- Minimal Accuracy Loss: On CIFAR-10, the clean accuracy only dropped by 6.24%, whereas other robust training methods often see much steeper declines.
- Speed: Meta-AT can adapt to a new attack in roughly 0.33 minutes for MNIST and 2 minutes for CIFAR-10, making it viable for real-world applications where threats evolve rapidly.
Conclusion
The MAD benchmark and Meta-AT algorithm provide a new paradigm for adversarial defense. By viewing defense as an agile adaptation process rather than a static shield, Meta-AT overcomes the traditional trade-offs of computational cost and generalization. Future work will focus on reducing dataset size and extending this framework to complex vision tasks like object detection and semantic segmentation.
Summary updated