Understanding and Improving Fast Adversarial Training

Maksym Andriushchenko, Nicolas Flammarion

TL;DR

Fast adversarial training using the Fast Gradient Sign Method (FGSM) is notoriously prone to "catastrophic overfitting," where a model suddenly loses all its robustness in a single training epoch. This paper reveals that this failure stems from a sudden breakdown in the model's local linearity, and introduces GradAlign, a novel regularizer that prevents this collapse and enables fast, single-step adversarial training even for large perturbation sizes.

What Problem Does This Paper Address?

Machine learning models are highly vulnerable to adversarial examples—tiny, worst-case perturbations that fool the model into making incorrect predictions. The most effective defense is adversarial training (AT), which trains the model on these adversarial examples. However, finding strong adversarial examples during training usually requires multi-step optimization methods like Projected Gradient Descent (PGD), which are computationally expensive and drastically slow down training.

To speed things up, researchers have tried using a single-step approximation called the Fast Gradient Sign Method (FGSM). Unfortunately, FGSM training suffers from a bizarre phenomenon called catastrophic overfitting: the model gains robustness for a few epochs, but then suddenly and completely loses it.

Catastrophic overfitting during FGSM training

Previous work suggested that adding a random initialization step (FGSM-RS) solved this by diversifying the types of adversarial examples seen. This paper challenges that explanation, showing that FGSM-RS still fails for larger perturbations, and seeks to uncover the true mathematical reason behind catastrophic overfitting to propose a more robust fix.

Methods

The authors first mathematically and empirically deconstruct the previous state-of-the-art (FGSM-RS). They show that adding a random step doesn't magically diversify the threat model; it simply reduces the average magnitude of the perturbation, keeping it within a smaller radius where the model behaves linearly.

To understand why FGSM fails, the authors introduce a metric called gradient alignment, which measures the local linearity of the loss function. It calculates the cosine similarity between the gradient at an input image and the gradient at a randomly perturbed point nearby.

Because FGSM relies on a linear approximation of the loss landscape to find adversarial examples, it requires high gradient alignment. The authors discover that catastrophic overfitting is exactly the moment this alignment collapses. To fix this, they introduce GradAlign, a regularization term added to the training loss. GradAlign explicitly maximizes the cosine similarity between gradients within the perturbation bounds, forcing the loss landscape to remain locally linear and ensuring the single-step FGSM attack remains a highly accurate approximation of a true worst-case adversary.

Datasets

  • CIFAR-10: Used for the primary experiments evaluating robustness across different perturbation radii ($\epsilon$).
  • SVHN (Street View House Numbers): Used to validate that the findings generalize to other standard image datasets.
  • ImageNet: Briefly explored to test scaling, though the authors note catastrophic overfitting behaves differently (or doesn't occur at standard $\epsilon$ values) at this scale.

Key Insights and Ideas

  • Randomness isn't the cure: The success of previous "random step" FGSM methods is just a side effect of taking a smaller step size. If you simply reduce the step size of standard FGSM without adding randomness, you get the exact same robustness benefits.
  • Catastrophic overfitting is a phase transition in linearity: When catastrophic overfitting occurs, the gradients inside the perturbation ball suddenly become nearly orthogonal (gradient alignment drops from ~0.95 to ~0.05). The model becomes highly non-linear locally, making the linear FGSM attack completely blind to the true adversarial vulnerabilities.
  • Even a single filter can break AT: Catastrophic overfitting isn't just a deep neural network mystery. The authors demonstrate it in a single-layer CNN, showing that a single filter learning an edge-detecting "Laplace" pattern can amplify noise so severely that it destroys local linearity and triggers catastrophic overfitting.
  • Initialization is a free lunch (temporarily): Standard neural network initialization schemes naturally yield highly aligned gradients. This explains why FGSM adversarial training always seems to work well at the very beginning of training before eventually collapsing.

Key Results

  • Prevents Overfitting for Large Perturbations: On CIFAR-10, for a large perturbation radius of $\epsilon = 16/255$, standard FGSM and FGSM-RS suffer catastrophic overfitting, dropping to ~0% robustness. FGSM + GradAlign maintains high robustness, achieving results comparable to the much slower multi-step PGD training.
GradAlign prevents catastrophic overfitting at large perturbation sizes
  • Consistent Across Datasets: On SVHN, GradAlign successfully prevents catastrophic overfitting up to $\epsilon = 12/255$, whereas standard fast AT methods fail early on.
  • Maintains Standard Accuracy: Unlike early stopping (a common hack to avoid catastrophic overfitting that severely hurts standard accuracy), GradAlign achieves the same high standard accuracy as full PGD-10 training.

Limitations and Future Work

  • Computational Overhead: Because GradAlign requires calculating the gradient of a gradient (double backpropagation), it introduces a roughly 3x slowdown compared to vanilla FGSM training. While still faster than 10-step PGD, the authors suggest future work should look for regularizers that stabilize gradients without needing double backprop.
  • ImageNet Dynamics: The authors note that catastrophic overfitting didn't naturally occur on ImageNet for standard perturbation sizes, making GradAlign unnecessary there. A deeper investigation into why ImageNet avoids this phenomenon is left for future work.
  • Robust vs. Catastrophic Overfitting: The authors explicitly note that GradAlign solves catastrophic overfitting (a sudden drop to zero robustness) but does not solve robust overfitting (a slow, gradual decline in test robustness over many epochs), which remains a separate open challenge in the field.

Summary updated

Abstract

A recent line of work focused on making adversarial training computationally efficient for deep learning models. In particular, Wong et al. (2020) showed that $\ell_\infty$-adversarial training with fast gradient sign method (FGSM) can fail due to a phenomenon called catastrophic overfitting, when the model quickly loses its robustness over a single epoch of training. We show that adding a random step to FGSM, as proposed in Wong et al. (2020), does not prevent catastrophic overfitting, and that randomness is not important per se --- its main role being simply to reduce the magnitude of the perturbation. Moreover, we show that catastrophic overfitting is not inherent to deep and overparametrized networks, but can occur in a single-layer convolutional network with a few filters. In an extreme case, even a single filter can make the network highly non-linear locally, which is the main reason why FGSM training fails. Based on this observation, we propose a new regularization method, GradAlign, that prevents catastrophic overfitting by explicitly maximizing the gradient alignment inside the perturbation set and improves the quality of the FGSM solution. As a result, GradAlign allows to successfully apply FGSM training also for larger $\ell_\infty$-perturbations and reduce the gap to multi-step adversarial training. The code of our experiments is available at https://github.com/tml-epfl/understanding-fast-adv-training.
BibTeX
@inproceedings{andriushchenko2020understanding,
  author = {Maksym Andriushchenko and Nicolas Flammarion},
  title = {{Understanding and Improving Fast Adversarial Training}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=va7hQlP1hwi},
}
Figure