TL;DR
This paper reveals that Adversarial Training (AT) suffers from "catastrophic forgetting," a problem usually associated with Continual Learning, where a model forgets how to defend against earlier adversarial examples as it encounters new ones. To fix this, the authors introduce Adaptive Multi-teacher Self-distillation (AMS), a method that uses previous versions of the model as "teachers" to preserve old knowledge, significantly boosting robustness and reducing the common issue of robust overfitting.
What Problem Does This Paper Address?
Deep Neural Networks are notoriously fragile when faced with adversarial examples—input data with tiny, intentional perturbations that cause the model to fail. While Adversarial Training (AT) is the most effective defense, it faces two major hurdles:
- The Robustness Gap: There is still a significant performance gap between "clean" accuracy (on normal data) and "robust" accuracy (on attacked data).
- Robust Overfitting: Unlike standard training, where more training usually helps, AT models often reach a peak in robustness and then start performing worse as training continues.
The authors hypothesize that these issues stem from the "minimax" nature of AT. Because the adversarial examples are generated based on the current state of the model, the data distribution shifts every epoch. The authors argue that AT is essentially a Continual Learning (CL) problem where the model is constantly learning new "tasks" (new distributions of harder attacks) and, in the process, suffers from catastrophic forgetting of the defensive strategies it learned earlier.
Methods
The authors propose a framework called Adaptive Multi-teacher Self-distillation (AMS). The core idea is to treat previous versions of the model as a committee of teachers that guide the current model.
- Stage-based Training: Training is divided into stages (e.g., every 20 epochs). At the end of each stage, the model's parameters are saved to a memory buffer.
- Multi-teacher Distillation: During the current training stage, the model tries to match its output probabilities (logits) with those of all previous "teacher" models stored in the buffer. This encourages the model to retain the defensive knowledge it acquired in earlier stages.
- Reweighting-based Loss Correction (RLC): This is the "Adaptive" part. Since adversarial training is hard, early-stage teachers are often wrong. Blindly following a "bad" teacher would hurt performance. AMS calculates a weight for each teacher based on its confidence in the true label of a sample. If a teacher misclassifies a sample, its influence on the loss function is minimized.
Datasets
The researchers evaluated their method on four standard computer vision benchmarks:
- CIFAR-10 & CIFAR-100: Small-scale image classification.
- SVHN (Street View House Numbers): Digit recognition in the wild.
- TinyImageNet: A subset of ImageNet with 200 classes at 64x64 resolution.
They tested these across different perturbation models ($\ell_\infty$ and $\ell_2$ norms) and various architectures, including PreActResNet-18, WideResNet-34-10, Vision Transformers (ViT), and VGG16.
Key Insights and Ideas
- Empirical Proof of Forgetting: The authors conducted a clever experiment: they collected adversarial samples that the model could correctly classify at Stage 1. By Stage 5, the model's accuracy on those same fixed samples dropped significantly (e.g., from 100% to ~75% on TinyImageNet). This confirms that the model "forgets" how to handle simpler attacks as it chases harder ones.
- AT as a "CL-style" Problem: By framing AT as a sequence of evolving tasks, the authors bridge two previously distinct fields. This allows them to borrow successful strategies from Continual Learning (like replay and distillation) to solve robustness problems.
- The "Bad Teacher" Problem: In standard Knowledge Distillation, the teacher is usually a superior, pre-trained model. In self-distillation for AT, the teachers are just "younger" versions of the student. The RLC mechanism is crucial because it allows the student to "disobey" the teacher when the teacher is clearly confused.
Key Results
- Improved Robustness: AMS consistently improved the "Auto Attack" (AA) accuracy—the gold standard for robustness—across all datasets. For example, on CIFAR-10, it boosted TRADES (a popular AT method) from 53.00% to 54.31%.
- State-of-the-Art on TinyImageNet: On the challenging TinyImageNet dataset, their method achieved 31.52% robust accuracy under Auto Attack, a significant result for this benchmark.
- Mitigating Overfitting: One of the most striking results was the reduction in robust overfitting. In standard TRADES, the gap between the "best" and "final" robust accuracy was 3.28%. With AMS, this gap shrank to just 0.58%, meaning the model remains stable even with extended training.
- Plug-and-Play: AMS proved to be a "unified enhancer," improving various base algorithms like PGD-10, TRADES, and AWP.
Limitations and Future Work
- Computational Overhead: Storing multiple teacher models and performing forward passes through them increases memory usage and training time. While the authors suggest an interval of $m=20$ epochs to balance this, it is still more expensive than vanilla AT.
- Theoretical Depth: While the empirical evidence for forgetting is strong, the authors note that the theoretical connection between the "forgetting" phenomenon and "robust overfitting" deserves deeper exploration.
- Scaling to ImageNet: The experiments stopped at TinyImageNet; testing the scalability of maintaining multiple teachers on full-scale ImageNet (1000 classes) remains a challenge for future research.
Summary updated