Adversarial training makes a classifier resistant to small, deliberately chosen input perturbations. It does this by training on attacked inputs, and it is expensive. The standard version finds each attack with $K$ steps of gradient ascent, so every weight update costs roughly $K$ extra forward and backward passes. The cheap version uses a single gradient step (the Fast Gradient Sign Method, FGSM). It has a well-known failure called catastrophic overfitting: partway through training, the model abruptly becomes robust only to the single-step attack it trains against, and its accuracy under real multi-step attacks falls to zero. For convolutional networks, FrequencyLowCut Pooling – Plug & Play against Catastrophic Overfitting traced this failure to a frequency artifact, aliasing in downsampling layers, and fixed it at 1.26× training cost. Vision Transformers have no downsampling layers, so that diagnosis and its fix do not carry over.
Transformers have their own frequency pathology. Self-attention averages tokens, and stacking such layers smooths token representations toward their mean ("over-smoothing" or rank collapse). We propose to test whether catastrophic overfitting in Vision Transformers has a measurable signature in this token-smoothing process: a sudden shift in how much token-to-token variation, the "high-frequency" part of the representation, survives in deep layers under attack. If it does, a one-scalar-per-layer regularizer that keeps that variation the same on clean and attacked inputs could keep single-step training from collapsing. The added cost would be about one forward pass, with no multi-step attack.
Core Claim
Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis splits each layer's token matrix $Z_l \in \mathbb{R}^{N \times d}$ ($N$ tokens, width $d$) into two parts. The "DC" part is the mean token. The "high-frequency" part is the residual after subtracting that mean. The paper shows attention shrinks the residual with depth. We track its share of energy,
$$h_l(x) = \frac{\| Z_l - \tfrac{1}{N}\mathbf{1}\mathbf{1}^\top Z_l \|_F^2}{\| Z_l \|_F^2},$$
where $\mathbf{1}$ is the all-ones vector and $\|\cdot\|_F$ is the Frobenius norm. Our first claim is empirical. During FGSM training of a Vision Transformer, the gap $|h_l(x+\delta) - h_l(x)|$ in deep layers (with $\delta$ the training perturbation) changes sharply at or before the epoch where multi-step robustness collapses.
Our second claim is that penalizing this gap prevents the collapse:
$$\mathcal{L} = \mathrm{CE}(f_\theta(x+\delta), y) + \lambda \sum_{l \in \mathcal{D}} \big(h_l(x+\delta) - h_l(x)\big)^2,$$
where $\mathcal{D}$ is the set of deep layers and $\lambda$ is a weight. We expect this to give FGSM-trained Vision Transformers multi-step robustness close to multi-step adversarial training, at about 1.3× the cost of plain FGSM.
The penalty is two-sided on purpose. It neither boosts high frequencies, as AttnScale does, nor removes them, as FLC pooling does. The literature disagrees about which direction is safe. Wang et al. amplify high-frequency token content to fight collapse. Enhancing Transferable Adversarial Attacks on Vision Transformers through Gradient Normalization Scaling and High-Frequency Adaptation shows Vision Transformers are especially vulnerable to perturbations aimed at high-frequency image bands. FLC pooling succeeds by cutting high frequencies. Holding the spectral profile fixed sidesteps that disagreement. The experiments also test amplification and cutting directly.
Research Program
The first stage is diagnosis. We train DeiT-S-style Vision Transformers with FGSM plus random initialization on CIFAR-10 and CIFAR-100 at $\ell_\infty$ radius 8/255. We use several step sizes, chosen to produce catastrophic overfitting reliably. Every few hundred iterations we log four signals:
- the per-layer $h_l$ on clean and attacked inputs;
- attention entropy;
- the largest pre-softmax attention logit;
- multi-step robust accuracy on a held-out batch.
Signals 2 and 3 matter because Towards Understanding and Improving Adversarial Robustness of Vision Transformers found that large pre-softmax values cause floating-point underflow. That underflow hides gradients and makes attacks look weaker than they are. Catastrophic overfitting in Transformers could be this same saturation rather than smoothing. Signal Propagation in Transformers further shows that under rank collapse, query and key gradients vanish. That gives a concrete way smoothing and gradient masking could be linked.
The diagnostic question is whether the $h_l$ gap is an earlier and more specific warning than the other signals. We measure this with lead time and with detection accuracy across runs that do and do not collapse.
The second stage is the intervention. We add the spectral-gap penalty and compare it against several alternatives:
- full-feature matching on the same layers;
- spectral amplification, using AttnScale or PIDformer as the architecture;
- spectral cutting, by low-passing the input patches as FLC does.
The third stage scales the best variant to ImageNet-100, which Jain and Dutta also use. There we check whether the gains survive larger images and more tokens.
Baselines
The naive reference is plain FGSM with random initialization on the same Vision Transformer. Following FLC pooling's convolutional result, we expect its multi-step robust accuracy to collapse to roughly 0%. FrequencyLowCut Pooling reports 0.00% PGD $\ell_\infty$ robust accuracy for baseline FGSM training on CIFAR-10. For Vision Transformers, no reported result was found on Lacuna for FGSM training. A second naive point is FGSM with early stopping on a held-out multi-step attack. This is the zero-cost way to avoid collapse, and the regularizer must exceed its final robust accuracy.
The strongest existing method in this regime is multi-step adversarial training on the same Vision Transformer. We run it two ways: plain PGD-10 adversarial training, and the AAS-AT variant of Jain and Dutta, which reports about 3.47% higher AutoAttack robustness than PGD adversarial training on CIFAR-10. No absolute Vision Transformer robust-accuracy number was found on Lacuna. We therefore rerun both in our pipeline with the same architecture, data augmentation, epoch count, and hyperparameter-search budget. We report robust accuracy against measured wall-clock cost.
The strongest fast method is FLC pooling, which reaches 38.41% PGD $\ell_\infty$ accuracy on CIFAR-10 with FGSM training at 1.26× cost. That number is for convolutional networks and cannot be compared directly with ours. For a like-for-like comparison we also apply FLC-style low-pass filtering to the patch embedding of our Vision Transformer.
Two ablations isolate the key ingredient:
- Feature matching. We replace the scalar spectral gap with full feature matching between clean and attacked tokens at the same layers. This is a training-time version of the invariance objective in Robustness Tokens.
- Architecture only. We FGSM-train AttnScale and PIDformer architectures with no regularizer.
The proposal "wins" if three things hold. It must prevent catastrophic overfitting in every seed and step size where plain FGSM collapses. Its AutoAttack accuracy must be within a few points of PGD-10 training at under half its wall-clock cost. And it must match or beat both ablations, since if feature matching or architecture alone does as well, the spectral framing is unnecessary.
Why not just…?
Why not just use multi-step adversarial training with fewer steps, or stop FGSM training early? These are the honest default, and the baselines include both. Early stopping keeps whatever robustness existed before the collapse. If the model collapses early, that is little. Two- or three-step attacks cost 3–4× FGSM. The regularizer earns its place only if it beats early stopping on final robustness and beats few-step attacks on cost. The cost-versus-AutoAttack curve will show directly whether either happens.
Why not just freeze a pretrained model and add a cheap defense like Robustness Tokens? Robustness Tokens recovers DINOv2-G white-box accuracy from 0.3% to 63.1% in 200 steps. That is very cheap. However, the robustness depends on the tokens staying secret, so it does not hold against an attacker who knows the defense. Our setting is attacker-aware robustness of a model that is itself trained, which this approach does not provide. We do include its invariance loss as the feature-matching ablation. If it beats the spectral penalty inside training, that is a real threat to our framing, and we would report it as such.
Why not just add an anti-oversmoothing architecture such as PIDformer and train with FGSM? This is the most serious alternative. PIDformer already improves FGSM accuracy on ImageNet from 33.64% to 38.52% with no adversarial training at all. It was never evaluated under multi-step attacks or AutoAttack, and never combined with fast adversarial training. We run it as an ablation. If it alone prevents catastrophic overfitting, the paper's contribution becomes that finding plus the diagnostic, and the regularizer becomes optional.
Why not just low-pass the input, as FLC pooling does? The high-frequency-attack paper suggests that removing input high frequencies could remove the bands Vision Transformers are most vulnerable to, which would make our method redundant. On the other hand, Wang et al. show deep Vision Transformer layers already lose high-frequency content, so further cutting may worsen over-smoothing and clean accuracy. The patch-embedding FLC baseline settles this empirically. Its clean accuracy and its robustness to token-level attacks are the deciding numbers.
Decisive Experiments
All robustness numbers use PGD-50 and AutoAttack, and we add the AAS-enhanced attack of Jain and Dutta to rule out gradient masking. Without that third attack, an apparent fix for catastrophic overfitting could simply be masking. We also evaluate against the high-frequency DCT attack of Zhang et al. and the block-sparse patch attack of Joshi et al. These test whether holding the spectral profile fixed opens a high-frequency weakness.
Cost is reported as wall-clock time and FLOPs relative to plain FGSM, on identical hardware. Three checks decide the outcome:
- Diagnostic check. The $h_l$ gap must give at least several hundred iterations of warning before collapse, and must outperform attention entropy and logit-magnitude signals at telling collapsing runs from stable ones on CIFAR-10. If it does not, the hypothesis is wrong. The diagnostic results still settle whether catastrophic overfitting in Transformers is mainly an attention-saturation effect.
- Intervention check. The penalty must prevent collapse across seeds and step sizes on CIFAR-10, CIFAR-100, and ImageNet-100, while staying near PGD-10 training on AutoAttack.
- Ablation check. The two-sided spectral penalty must beat the amplification, cutting, and full-feature-matching variants. This is what shows the spectral profile, rather than generic feature consistency, is the active ingredient.
The same logged runs also inform two open questions from the literature: whether representation over-smoothing helps or hurts robustness, and whether the last-layer contraction reported by Probing the Embedding Space of Transformers via Minimal Token Perturbations is harmful smoothing or useful convergence.
References
- FrequencyLowCut Pooling – Plug & Play against Catastrophic Overfitting
- Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice
- Enhancing Transferable Adversarial Attacks on Vision Transformers through Gradient Normalization Scaling and High-Frequency Adaptation
- Towards Understanding and Improving Adversarial Robustness of Vision Transformers
- Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse
- PIDformer: Transformer Meets Control Theory
- Robustness Tokens: Towards Adversarial Robustness of Transformers
- Adversarial Token Attacks on Vision Transformers
- Probing the Embedding Space of Transformers via Minimal Token Perturbations