Adversarial training makes a classifier robust by training it on worst-case inputs, but finding those inputs usually takes many gradient steps per training example. Cheap one-step versions exist, and they share a known failure: partway through training the model suddenly learns to fool the one-step attacker while staying fully vulnerable to stronger attacks. This failure is called catastrophic overfitting. Everything we know about it comes from attacks that add small pixel noise. The attacks that matter in physical and human-supervised settings are different. They are "stealthy": they change an image through a generative model's latent space, adding things like rust, makeup, or paint, so the result still looks natural. We ask whether one-step adversarial training in a generator's latent space also collapses. If it does, we ask whether a zero-cost trick, which perturbs all of the generator's per-layer style inputs at once using gradients the single step already computes, prevents the collapse.
The question matters because defenses against stealthy attacks are currently either expensive or unvalidated. The only training-based on-manifold defense in our file searches for pairs of latent codes iteratively and was shown on MNIST and CelebA. The natural-damage attack AdvWT improves accuracy on real damaged traffic signs when used for training, but it relies on slow Monte Carlo search. Making these defenses cheap requires single-step latent attacks. There is also a specific reason to expect those attacks to fail. Prior work shows that catastrophic overfitting is triggered by easy, discriminative, low-frequency features whose size is close to the attack budget, and stealthy perturbations are low-frequency almost by construction.
Core Claim
We claim three things. First, single-step adversarial training in a frozen generator's latent space shows catastrophic overfitting: a sudden collapse of robustness against multi-step latent attacks while robustness against the single-step training attack keeps rising. Second, the trigger can be predicted. Collapse happens when the data contain a label-correlated direction inside the generator's range whose size is comparable to the latent budget, which carries the feature account of Catastrophic overfitting is a bug but it is caused by features onto the image manifold. Third, the known cheap fixes from pixel-space work do not carry over. Frequency filtering does not help because the harmful features are themselves low-frequency. What does help is a latent analogue of single-step latent adversarial training (SLAT): one step that perturbs every layer-wise style code of the generator jointly, each with its own budget. It uses gradients already produced by the single backward pass, so it adds no extra passes.
Positioning Against Prior Work
Several papers supply the diagnosis and constrain the design. Fast is better than free introduced one-step training with random starts and named catastrophic overfitting. Understanding and Improving Fast Adversarial Training tied the collapse to a loss of local linearity: the cosine between nearby input gradients falls from about 0.95 to about 0.05. It fixed this with GradAlign at the price of double backpropagation. Layer-Aware Analysis of Catastrophic Overfitting placed the "pseudo-robust shortcuts" in early layers. FrequencyLowCut Pooling and the low-pass result of Ortiz-Jimenez et al. remove high frequencies. All of these work inside an $L_\infty$ pixel box. We reuse their diagnostics, gradient alignment and Hessian curvature, but measure them in latent coordinates. We expect the frequency fixes to fail in this setting, and we test that expectation directly rather than assuming it.
Our mechanism comes from Reliably fast adversarial training via latent adversarial perturbation. SLAT perturbs hidden layers of the classifier using gradients that are already free, and this acts as an implicit regularizer of local linearity. Our variant perturbs the generator's layer-wise style codes instead, for example StyleGAN's per-resolution codes or StarGAN-v2 style codes. Those perturbations stay on the image manifold, while SLAT's do not. Towards Speeding up Adversarial Training in Latent Spaces also perturbs a latent space, the classifier's logits, but it never checks for collapse under adaptive attacks and never produces natural images.
The Robust Manifold Defense is the closest work. It already performs adversarial training through a generator. It uses iterative pair search, tests nothing single-step, and shows that projecting inputs onto the manifold, as Defense-GAN does, is broken by on-manifold attacks; that last result rules out projection as an alternative to our approach. The stealth attacks CLIP2Protect and Adversarial Wear and Tear serve here as the threat models and as strong evaluation attacks.
Research Program
The setting is a frozen generator $G$ with layer-wise codes $w = (w_1,\dots,w_L)$. Training images come either from sampled codes or from real images inverted once and cached, so the cost of inversion is paid only once. The latent threat model allows $\|w'_\ell - w_\ell\| \le \epsilon_\ell$ for every layer $\ell$. The classifier $f_\theta$ is trained on
$$\min_\theta \; \mathbb{E}_{(w,y)} \max_{\|\delta_\ell\|\le\epsilon_\ell} \mathcal{L}\big(f_\theta(G(w+\delta)), y\big),$$
where $\delta = (\delta_1, \dots, \delta_L)$ is the latent perturbation. The inner maximum is approximated by one signed gradient step. We compare three ways to place that step: a single shared code, only the coarse layers, and all layers jointly with separate budgets. Our proposed method is the all-layer version.
The work proceeds in three parts. The first is phenomenology. We sweep $\epsilon$ and record multi-step latent robust accuracy every epoch, together with latent gradient alignment and the largest Hessian eigenvalue taken with respect to $w$. The question is whether collapse happens and whether the same linearity signature precedes it.
The second part is a causal test. We inject a synthetic label-correlated direction $v(y)$ into the latent codes as $w + \beta v(y)$, chosen once in coarse layers (low-frequency, semantic) and once in fine layers. We then check whether collapse peaks at $\beta \approx \epsilon$, as the pixel-space theory predicts. We also measure the image-space spectrum of $G(w + \beta v(y)) - G(w)$, which tells us whether low-pass filtering could even reach the feature.
The third part is the fix and its accounting. We report wall-clock time per epoch for every method on the same GPU. The one-step attack costs one extra forward and backward pass through $G$ and $f$. The all-layer perturbation adds nothing beyond that, while latent GradAlign adds a double backward through $G$.
Baselines
The naive reference is standard training on generator outputs. Next to it is one-step latent training with a random start and early stopping chosen on a multi-step latent attack, which is the direct port of Fast is better than free. It is the cheapest serious competitor. We beat it if we hold robustness at budgets where it collapses, without having to stop early.
The strongest existing method is multi-step latent adversarial training in the style of the Robust Manifold Defense, with 5 and 10 steps. That paper reports 96.26% accuracy under a white-box $L_2 = 1.5$ attack on MNIST, against 91.88% for TRADES. On CelebA it reports only that clean accuracy stays at 90%; no reported latent robust accuracy was found on Lacuna for CelebA. We will therefore rerun it ourselves on identical cached codes, with the same classifier, epochs, and tuning budget. Our method counts as a success if it reaches at least 90% of that method's multi-step latent robust accuracy at less than a third of its wall-clock cost. On traffic signs, AdvWT-based training raised accuracy on real damaged signs from 90.1% to 95.4% (Adversarial Wear and Tear). Our one-step method should match that gain while doing no Monte Carlo search during training.
We also include pixel-space fast defenses at matched cost: SLAT (47.06% under PGD-50 on CIFAR-10 in its paper), LAP, and FGSM training on low-pass-filtered data. These test whether pixel robustness alone already covers stealthy attacks.
Two ablations isolate the key ingredient. One perturbs a single code instead of all layers. The other replaces the adversarial direction with a random one at the same budgets. Latent GradAlign serves as the expensive upper reference for preventing collapse.
Why not just…?
Why not just augment with random latent perturbations? Random style sampling is cheap and gave AdvWT its practical benefit, so this alternative is a real threat. The random-direction ablation settles it. If random perturbations at matched budgets come within a few points of adversarial ones under multi-step latent attacks, then the adversarial step is unnecessary and we will report that. We expect a gap because worst-case directions in the latent space are rare under random sampling, but this is something we will measure, not assume.
Why not just run two or three latent PGD steps, since the latent space is small? The small latent dimension does not make each step cheap. Every step requires a full backward pass through the generator, which costs as much as or more than the classifier. The wall-clock accounting will show whether few-step PGD reaches its robustness more cheaply than our method. If it does, our contribution shrinks to the diagnosis of collapse.
Why not just use a pixel-space fast defense, or low-pass filtering? Low-pass filtering and FrequencyLowCut pooling remove high frequencies. Stealthy features live at low frequencies, and the causal test measures exactly where the injected features fall in the spectrum. Pixel-robust models from SLAT and LAP will be evaluated under the latent attacks. If they are already robust there, the latent threat model is redundant, and that would be a useful negative result.
Why not just apply GradAlign in latent space? It probably works, and we include it as an upper bound. Its double backward pass through a StyleGAN-sized generator is the overhead this project is trying to avoid.
Proposed Experiments
The main testbeds are CelebA attribute classification with a StyleGAN2 generator, using cached inversions of real images, and GTSRB-HQ plus the 4,529-image Wear and Tear set with StarGAN-v2 style codes, following AdvWT. MNIST with a small generator allows a direct comparison with the Robust Manifold Defense. Classifiers are ResNet-18 and a small Vision Transformer, the latter because the frequency fixes were validated only on convolutional networks.
The metrics are clean accuracy and multi-step latent robust accuracy (50 steps, 5 restarts, per-layer budgets), plus AdvWT-style Monte Carlo attack success and pixel-space AutoAttack following RobustBench to check for gradient masking. We also report latent gradient alignment, latent curvature, and wall-clock time. All figures are averaged over 5 seeds, because collapse is erratic.
Three checks decide the outcome. First, one-step latent training must show collapse at some budget, meaning multi-step robustness near zero while single-step robustness rises; otherwise the premise fails and the paper becomes a negative result about stealthy fast training. Second, injecting a coarse-layer feature at $\beta \approx \epsilon$ should cause collapse that low-pass filtering does not prevent, while a fine-layer feature at the same scale should be suppressed by the filter. Third, all-layer joint perturbation should prevent collapse across the budget sweep at the per-epoch cost of plain one-step training, and the gap to the random-direction ablation should exceed seed variance.
References
- Catastrophic overfitting is a bug but it is caused by features
- Fast is better than free: Revisiting adversarial training
- Understanding and Improving Fast Adversarial Training
- Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency
- FrequencyLowCut Pooling – Plug & Play against Catastrophic Overfitting
- Reliably fast adversarial training via latent adversarial perturbation
- Towards Speeding up Adversarial Training in Latent Spaces
- The Robust Manifold Defense: Adversarial Training using Generative Models
- CLIP2Protect: Protecting Facial Privacy Using Text-Guided Makeup via Adversarial Latent Search
- Adversarial Wear and Tear: Exploiting Natural Damage for Generating Physical-World Adversarial Examples
- RobustBench: a standardized adversarial robustness benchmark