Adversarial training makes an image classifier resistant to small, deliberately crafted input perturbations by training it on perturbed examples. It has two well-known costs. It is slow, because finding a strong perturbation takes many gradient steps per update. It also lowers accuracy on ordinary, unperturbed images. The cheap version, which takes a single gradient step to build each perturbation, often fails partway through training. Robustness against stronger multi-step attacks suddenly falls to near zero, while the model still looks robust to the single-step attack it trains on. This failure is called catastrophic overfitting. Reinforcement learning researchers met an analogous problem: Q-values that blow up and training runs that fail on some random seeds and not others. They fixed it with a one-line change to the network. They rescaled the second-to-last layer's feature vector to unit length before the final linear layer.
This note proposes porting that change to single-step adversarial training and testing it as a mechanism, not just as a trick. Projecting the penultimate features onto the unit sphere removes one way the network can change its outputs: making a feature vector longer. With that option gone, the network can only change the direction of its features. We ask whether this one constraint, which adds no extra backward passes, cuts the rate of catastrophic overfitting across seeds and shrinks the clean-accuracy penalty. We also ask where it stops working. The answer matters because the existing fixes for single-step failure all add measurable compute, and none of them addresses the clean-accuracy drop.
Core Claim
Our claim has two parts. First, in single-step adversarial training, part of the instability and part of the clean-accuracy loss travel through the length of the penultimate feature vector. Second, a fixed-scale normalized head removes that part at zero extra cost. The head we study computes logits as
$$\ell(x) = s\, W \frac{z(x)}{\|z(x)\|_2},$$
where $z(x)$ is the backbone's penultimate feature vector, $W$ is the usual trainable linear classifier, and $s$ is a fixed scalar that we treat as a deliberate knob. The input gradient that the single-step attack uses then factors as
$$\nabla_x \mathcal{L} = \frac{s}{\|z\|}\, J(x)^\top \left(I - \hat z \hat z^\top\right) W^\top (p - y),$$
where $J(x)$ is the backbone Jacobian $\partial z / \partial x$, $\hat z = z/\|z\|$, $p$ is the softmax output, and $y$ is the one-hot label.
Two consequences follow, and both can be tested. First, perturbations that only lengthen or shorten the feature vector provably cannot change the prediction, and the gradient correctly ignores them. Second, feature length enters only as a scalar prefactor. A sign-based attack (FGSM) is invariant to that prefactor, so the network cannot make itself look robust to FGSM by rescaling its features. The fixed $s$ also caps how confident the logits can become. This matters because overconfident fitting of the training attack's perturbed points, accompanied by input-gradient norms rising up to 35x, is a documented signature of collapse.
The claim is deliberately falsifiable against a competing account. Layer-Aware Analysis of Catastrophic Overfitting finds that catastrophic overfitting begins in the early layers, through "pseudo-robust shortcuts" made of a few very large weights. If that account is complete, a change at the head should delay collapse at best. The proposal is built so that this outcome is informative rather than a failure: it would tell us which part of the collapse goes through the head and which does not.
How This Sits Relative to Prior Work
The normalization side comes from reinforcement learning, which supplies the method. Is High Variance Unavoidable in RL? shows that unit-normalizing the features before the final layer keeps the output nonlinearity out of its flat region and cuts run-to-run standard deviation by more than 3x across 21 control tasks. Dissecting Deep RL with High Update Ratios applies the same projection to the critic and keeps Q-values from diverging to $10^{10}$. Neither paper involves adversaries. We borrow two things from them: the operation itself, and their way of measuring success, which is variance across seeds rather than the best run.
Two papers already combine penultimate features with adversarial training, and our proposal differs from both. Domain Adaptation with Adversarial Training on Penultimate Activations perturbs normalized penultimate features in closed form, and finds that its normalized variant works better. However, it targets domain shift and never evaluates attacks in input space. Adversarial Feature Augmentation and Normalization perturbs late features and mixes in batch-normalization statistics. It improves clean accuracy, but gives only modest white-box robustness. We keep the attack in input space, where the threat model lives, and use normalization only as an architectural constraint.
Two results constrain the design. Batch Normalization Increases Adversarial Vulnerability reports that normalization throughout the network pushes models toward brittle features. That is why we confine normalization to one vector, and why we test a variant with batch normalization removed. Understanding and Improving Fast Adversarial Training shows that random initialization (FGSM-RS) only shrinks the effective perturbation radius and still collapses at 16/255. This tells us that any fix has to be tested at large radii.
Phase Transition from Clean Training to Adversarial Training motivates the accuracy half of the claim. It shows that clean accuracy collapses once the training radius exceeds a critical value $\epsilon^*$, which is about 0.5/255 for PreActResNet-18. Beyond that point, training lands in a different set of solutions. Wider models raise $\epsilon^*$. We ask whether the head constraint does the same.
Research Program
The work has three linked parts.
The first part builds the intervention and its ablations. We insert unit normalization on the penultimate features of PreActResNet-18 and WideResNet backbones and use the fixed scale $s$ as described above. We then train with the FGSM-RS recipe of Fast is better than free. The ablations separate four candidate ingredients: removing feature length, capping logit scale, normalizing the classifier weights, and changing the gradient's geometry. Each of these could be what actually helps.
The second part is diagnostic and runs alongside every training run. We log per-class feature length, logit norm, input-gradient norm, gradient alignment within the perturbation ball (the cosine that collapses from about 0.95 to about 0.05), and the fraction of very large early-layer weights that LAP identifies as shortcuts. These logs show whether collapse, when it still happens, starts early in the network or at the head. They also show whether normalization merely moves the instability elsewhere. The concern there is that the backbone could compensate by inflating its Jacobian $J$.
The third part measures the trade-off. We follow the phase-transition paper's procedure: train at a grid of small radii and locate the point where clean accuracy breaks away from clean training. We do this for both normalized and standard heads, as a function of $s$. If the normalized head raises $\epsilon^*$, that is direct evidence that it changes the trade-off rather than merely stabilizing training.
Baselines
The naive reference point is plain FGSM-RS on an unmodified network (Fast is better than free). It is reported to reach 45% PGD accuracy on CIFAR-10 at 8/255 in 6 minutes. The file does not report its clean accuracy on Lacuna, so we will measure it. It is also reported to collapse to roughly 0% at 16/255 (GradAlign). We run it with the identical schedule, mixed precision, and seeds.
The strongest fast competitor at large radii is LAP (Layer-Aware Analysis of Catastrophic Overfitting). It reports 19.55% AutoAttack accuracy on CIFAR-10 at 16/255, with about 7% overhead over FGSM, where FGSM collapses to 0.0%. This is the number to beat, because it attacks the same failure at nearly the same cost.
GradAlign is the strongest compute-heavier fast method. The file says it matches multi-step PGD robustness at 16/255 but gives no exact figure, so there is no reported AutoAttack number found on Lacuna for GradAlign at 16/255. We will run it ourselves.
As upper references we use PGD-10 and TRADES. For TRADES, Theoretically Principled Trade-off reports 56.61% PGD robust accuracy, with natural accuracy reaching 88.64% at weaker regularization. For multi-step training, RAT reports 49.19% AutoAttack.
Fairness rules apply to every method. All methods get the same epoch count, the same data, and an equal hyperparameter search of the same number of configurations. Methods are compared at matched wall-clock time on the same GPU. Every method uses the same 10 seeds.
"Beating" a baseline has a precise meaning here. At 16/255, the normalized head plus FGSM-RS must beat LAP's AutoAttack accuracy. Alternatively, it must match that accuracy at higher clean accuracy and lower cost, with a collapse rate across seeds no worse than LAP's. At 8/255, it must match FGSM-RS's robust accuracy while improving clean accuracy outside the seed-to-seed spread.
The ablation that isolates the key ingredient runs four variants:
- an unnormalized head whose logit norm is clipped to the same ceiling $s\|W\|$;
- a head whose classifier rows are normalized but whose features are not;
- normalized features with a learnable $s$;
- the full fixed-$s$ normalized head.
If the logit-clipping variant matches the full method, the mechanism is logit scale, not feature length, and we will report it that way.
Why not just…?
Why not just clip or penalize the logits, or use label smoothing? This alternative is a real threat. A capped logit scale may be the entire effect, and techniques that squash logits are known to produce gradient masking: they make attacks look weaker without making the model more robust. The logit-clipping ablation above tests the first possibility directly. To guard against masking, we evaluate only with AutoAttack under RobustBench protocols. Those protocols exposed robustness overestimates of more than 10% in published defenses, and they include the DLR loss, which does not depend on logit scale. If logit clipping matches our head, the proposal's contribution shrinks to a cheaper diagnosis of why fast training collapses, and we would say so.
Why not just use GradAlign or LAP? Both work at large radii, but neither is free. GradAlign differentiates through a product of gradients, and LAP adds weight-gradient perturbations at about 7% overhead. Neither targets clean accuracy. Our head adds no backward passes, and it composes with both. Head-plus-LAP is therefore a planned run: if LAP's early-layer account is right, the two should add up rather than overlap.
Why not just use early stopping, smaller steps, or a tuned schedule for FGSM-RS? Early stopping recovers the best checkpoint from runs that collapse, but it does not stop the collapse. It also needs a held-out multi-step attack to decide when to stop. More importantly, step-size tuning fails at 16/255 (GradAlign). We give FGSM-RS the same tuning budget, report its collapse rate across seeds at each radius, and report its tuned best.
Why not just remove batch normalization? Removing it raises intrinsic robustness but costs clean accuracy (66.5% vs 70.5% on ImageNet in Benz et al.), and it destabilizes training. We test it alone and combined with our head. The combination is the configuration most likely to show whether head normalization can replace the stability that batch normalization provides.
Proposed Experiments and Decisive Checks
Main experiments. We run CIFAR-10, CIFAR-100, and SVHN at $\ell_\infty$ radii of 8/255, 12/255, and 16/255, with 10 seeds per cell. For each run we report clean accuracy, AutoAttack accuracy, PGD-50 accuracy, wall-clock time, and the fraction of seeds that collapse, defined as PGD-50 accuracy falling below 5%. SVHN matters especially because RAT fails there.
Check 1: collapse rate. At 16/255, the normalized head must lower the collapse rate relative to FGSM-RS. If it does not, the main claim fails.
Check 2: where collapse starts. In runs that still collapse, the logged diagnostics must show whether early-layer weight growth precedes the change in head-side quantities. This distinguishes "the head constraint is insufficient" from "the head constraint moved the instability into the backbone Jacobian."
Check 3: the phase transition. For several values of $s$, we compare $\epsilon^*$ with and without normalization. A rise in $\epsilon^*$ would show that the constraint changes the robustness-accuracy trade-off, not just training stability.
Masking audit. We compare the gap between white-box and transfer attacks, and between gradient-based and black-box Square attacks, for each method. Any method whose Square-attack accuracy falls below its gradient-attack accuracy is flagged as masking gradients.
Scale test. If the CIFAR results hold, we repeat the Fast AT protocol on ImageNet with ResNet-50, where 43% PGD accuracy in 12 hours is the reported reference. The question is whether the head keeps clean accuracy higher at matched robustness.
References
- Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency
- Is High Variance Unavoidable in RL? A Case Study in Continuous Control
- Dissecting Deep RL with High Update Ratios: Combatting Value Divergence
- Domain Adaptation with Adversarial Training on Penultimate Activations
- Adversarial Feature Augmentation and Normalization for Visual Recognition
- Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspective
- Understanding and Improving Fast Adversarial Training
- Phase Transition from Clean Training to Adversarial Training
- Fast is better than free: Revisiting adversarial training
- Theoretically Principled Trade-off between Robustness and Accuracy
- On Intriguing Layer-Wise Properties of Robust Overfitting in Adversarial Training
- RobustBench: a standardized adversarial robustness benchmark