Abstract
A recent line of work focused on making adversarial training computationally efficient for deep learning models. In particular, Wong et al. (2020) showed that $\ell_\infty$-adversarial training with fast gradient sign method (FGSM) can fail due to a phenomenon called "catastrophic overfitting", when the model quickly loses its robustness over a single epoch of training. We show that adding a random step to FGSM, as proposed in Wong et al. (2020), does not prevent catastrophic overfitting, and that randomness is not important per se -- its main role being simply to reduce the magnitude of the perturbation. Moreover, we show that catastrophic overfitting is not inherent to deep and overparametrized networks, but can occur in a single-layer convolutional network with a few filters. In an extreme case, even a single filter can make the network highly non-linear locally, which is the main reason why FGSM training fails. Based on this observation, we propose a new regularization method, GradAlign, that prevents catastrophic overfitting by explicitly maximizing the gradient alignment inside the perturbation set and improves the quality of the FGSM solution. As a result, GradAlign allows to successfully apply FGSM training also for larger $\ell_\infty$-perturbations and reduce the gap to multi-step adversarial training.
The Problem: Catastrophic Overfitting in Fast Training
Deep learning models are notoriously vulnerable to "adversarial examples"—tiny, calculated perturbations that flip a model's prediction. To defend against this, researchers use Adversarial Training (AT), which involves training the model on these difficult examples. The gold standard for AT is Projected Gradient Descent (PGD), but it is computationally expensive because it requires multiple gradient steps for every training image.
The Fast Gradient Sign Method (FGSM) is a much faster, single-step alternative. However, FGSM training is plagued by catastrophic overfitting (CO): the model appears robust for several epochs, but then its robustness against multi-step attacks (like PGD) abruptly drops to 0% within a single epoch. This is often accompanied by a sudden spike in the adversarial loss calculated via PGD, even as the FGSM loss remains low.
Methods: GradAlign Regularization
The authors discover that CO isn't exclusive to deep networks; it can occur in a single-layer CNN with just one filter. The model essentially "cheats" the FGSM attack by developing sharp local non-linearities that make the gradient at the starting point a poor predictor of the loss surface nearby.
To counter this, they propose GradAlign. This is a regularization term added to the loss function that maximizes the gradient alignment (cosine similarity) between the gradient at the original input $x$ and the gradient at a nearby randomly perturbed point $x + \eta$:
$$ \Omega(x, y, \theta) = 1 - \cos(\nabla_x \ell(x, y; \theta), \nabla_x \ell(x + \eta, y; \theta)) $$
By forcing these gradients to be similar, GradAlign ensures the loss surface is locally linear. This makes the single-step FGSM attack an accurate approximation of the "worst-case" perturbation, preventing the model from exploiting local non-linearities to hide from the trainer.
Key Insights
- Randomness as a Reducer: Previous work suggested that adding a random step to FGSM (FGSM-RS) worked by "diversifying" the attack. This paper proves that the random step actually helps simply by reducing the average magnitude of the perturbation, making the linear approximation more likely to hold.
- The Single-Filter Culprit: CO can be triggered by a single convolutional filter that learns to amplify high-frequency noise. When this happens, the model's "local linearity" collapses.
- Breaking the $\epsilon$ Barrier: Standard FGSM training usually fails when the perturbation radius ($\epsilon$) is large. GradAlign allows FGSM to remain stable even at much higher $\epsilon$ values where other fast methods fail.
Key Results
The researchers evaluated GradAlign on CIFAR-10, SVHN, and ImageNet. A major finding is that GradAlign maintains stability across a wide range of perturbation radii, whereas other fast methods collapse as $\epsilon$ increases.
On CIFAR-10 with $\epsilon=8/255$, GradAlign achieved 47.58% robustness against a strong PGD-50-10 attack, significantly outperforming standard FGSM (~0%) and FGSM-RS (45.1%), and coming close to the much slower PGD-10 training (50.04%). At a larger radius of $\epsilon=16/255$, where all other fast methods collapsed to 0% robustness, GradAlign maintained 28.88% robustness.
Conclusion
GradAlign effectively bridges the gap between fast, single-step adversarial training and slow, multi-step methods. While it introduces a roughly 3x computational overhead compared to vanilla FGSM due to double backpropagation, it remains significantly faster than PGD-10 while providing robust protection and preventing the sudden collapse of model performance.
Summary updated