Adversarial training makes a classifier robust by training it on inputs that have been deliberately perturbed to fool it. Finding those perturbations is expensive. The standard recipe runs several gradient steps on each input before every weight update, which multiplies training cost roughly by the number of steps. The cheap alternative takes a single step from a random starting point. This single-step method is known as "fast adversarial training," and it cut CIFAR-10 robust training from 80 hours to 6 minutes (Fast is better than free). It has one failure that matters: catastrophic overfitting. Partway through training, robustness against strong multi-step attacks suddenly drops to near zero, while the model still resists the weak single-step attack it trains on. The best current fixes add curvature penalties, second-order heuristics, or historical perturbation buffers. Each costs extra computation or adds a sensitive hyperparameter.
This note proposes treating single-step adversarial training as a two-objective problem and reading catastrophic overfitting off the conflict between those two objectives. The single-step attack already runs one forward and backward pass at a randomly perturbed, nearly clean input. If that backward pass also computes weight gradients, we obtain a near-clean parameter gradient $g_r$ as a by-product, and it can be compared with the adversarial gradient $g_a$ from the actual training step. The claim has two parts. First, the angle between $g_r$ and $g_a$ is a cheap, first-order readout of the curvature quantity previously shown to explode just before collapse, so it should warn of collapse in advance. Second, removing the conflicting component of the update, in the style of multi-task gradient surgery, should prevent collapse. This fix costs no extra backward pass and adds no regularizer weight to tune.
Core Claim
Let $\delta_0$ be the random start and $\delta$ the single-step perturbation. The two gradients are $g_r = \nabla_\theta \mathcal{L}(f_\theta(x+\delta_0),y)$ and $g_a = \nabla_\theta \mathcal{L}(f_\theta(x+\delta),y)$, where $\theta$ are the network weights and $\mathcal{L}$ is the cross-entropy loss. A first-order Taylor expansion in the input gives
$$g_a - g_r \approx H_{\theta x}\,(\delta - \delta_0),$$
where $H_{\theta x} = \nabla_\theta \nabla_x \mathcal{L}$ is the mixed Hessian, which measures how moving the input changes the weight gradient. A Training-Dynamics View of Catastrophic Overfitting showed that a spike in the spectral norm of $H_{\theta x}$ comes just before catastrophic overfitting. They control it with a KL consistency penalty that needs extra forward passes. The relation above shows that the part of the adversarial update driven by this mixed curvature is exactly the part that can turn $g_a$ against $g_r$. The cosine $\cos(g_r, g_a)$, along with the relative norm $\|g_a - g_r\|/\|g_r\|$, therefore measures the dangerous curvature along the actual attack direction at almost no cost. Our hypothesis is that this cosine falls sharply and turns negative epochs before robust accuracy collapses.
The intervention is a conflict-aware update. When $\langle g_r, g_a\rangle \ge 0$ we use $g_a$ unchanged. Otherwise we use
$$d = g_a - \frac{\langle g_a, g_r\rangle}{\|g_r\|^2}\, g_r.$$
This removes from the adversarial step only the component that would raise the near-clean loss.
One geometric point should be stated directly. Harmonized Cone warns that conflict-free directions can be "infeasible": they may not be expressible as a non-negative mix of the objective gradients, and such updates can trap training. For exactly two objectives this trap does not arise here. When the inner product is negative, $d = g_a + c\,g_r$ with $c>0$, and $d$ has non-negative inner product with both gradients. So $d$ already lies in their harmonized cone, and the polyhedral machinery is unnecessary. The cone geometry matters only once a third term is added, for example the KL regularizer. We treat that three-objective case as an extension, not the core contribution.
Relation to Prior Work
The closest paper is Conflict-Aware Adversarial Training. It motivates this proposal: it shows that clean and adversarial gradients conflict more as the perturbation budget grows, and it projects the adversarial gradient onto a cone around the clean one. It works only with multi-step attacks, pays for a separate clean backward pass, and never examines single-step training or catastrophic overfitting. We differ in three ways. We take the reference gradient from the attack's own backward pass. We tie the conflict analytically to the mixed Hessian. We test whether surgery prevents collapse rather than whether it softens the clean-accuracy cost of robustness.
Gradient Surgery for Multi-Task Learning supplies the projection itself. Harmonized Cone supplies the feasibility argument that tells us when that projection is safe.
On the fast-training side, three methods are our competitors for preventing collapse. The mixed-Hessian KL regularizer (Training-Dynamics View) penalizes curvature directly. SORA uses previous-batch gradient alignment, called PertAlign, to adapt step size cheaply. FGSM-PCO blends current and historical perturbations. All three change the attack or the loss. We change neither: we reshape the weight update. That difference means the methods could be combined, and the combination is tested.
Fast is better than free constrains the design. Random initialization is part of our method because $\delta_0$ is what makes $g_r$ available at all.
Research Program
The work has three stages, each of which can stop the project.
The first stage is diagnostic. We train FGSM with random initialization at budgets where catastrophic overfitting is frequent, and log four quantities every few hundred iterations: $\cos(g_r,g_a)$, $\|g_a-g_r\|$, a power-iteration estimate of $\|H_{\theta x}\|_2$, and SORA's PertAlign. We then measure the lead time between each signal's crossing and the collapse of PGD-50 accuracy, using a held-out batch. We also check the Taylor relation directly by comparing $g_a-g_r$ with finite-difference Hessian-vector products. If the cosine gives no warning, the mechanism is wrong, and we report that negative result rather than continue.
The second stage is the intervention. We add the conflict-aware update to FGSM with random initialization and measure the collapse rate, robustness, and wall-clock time.
The third stage tests composition and scope. We add the KL regularizer as a third objective, where the harmonized-cone construction is needed, and we test ViT-Small, where single-step training is known to collapse.
The method is not literally free. Computing weight gradients during the attack's backward pass adds weight-gradient matrix products and memory for one extra gradient vector. We will report measured throughput rather than claim zero overhead, and compare it against the cost of a separate clean backward pass.
Baselines
The naive reference point is FGSM with uniform random initialization, as in Fast is better than free. That paper reports 45% robust accuracy on CIFAR-10 in 6 minutes. It collapses under longer training or larger budgets, and our method must remove that collapse at nearly the same cost. Multi-step PGD-10 training is the expensive reference. The Training-Dynamics paper reports 51.79% PGD-10 accuracy for it with ResNet-18 on CIFAR-10, and FGSM-PCO reports 53.19% in its own setup.
The strongest existing single-step methods are the following. FGSM-PCO reports 56.32% PGD-10 accuracy with ResNet-18 on CIFAR-10 and 0 collapses in 10 WideResNet34-10 runs on CIFAR-100, compared with 6 of 10 for FGSM-MEP (FGSM-PCO). The KL regularizer reports 52.46% PGD-10 accuracy at roughly 3x the speed of PGD-10, and 32.59% for ViT-Small (Training-Dynamics View). SORA reports 32.91% AutoAttack accuracy on PathMNIST, where FGSM and N-FGSM collapse (SORA). These numbers come from different setups, so we will rerun all methods ourselves with the same architecture, epochs, learning-rate schedule, data augmentation, and per-method tuning budget, and with 10 seeds each. CA-AT with its PGD inner loop serves as a costly upper reference for conflict-aware training. No CA-AT single-step result was found on Lacuna.
Two ablations isolate the key ingredient. The first is a static mix, $g_a + \lambda g_r$, which uses the same free gradient but applies no projection. The second is our projection computed against a true clean gradient from a separate backward pass. That run shows whether the random-start gradient is a good enough substitute for the clean one.
Our method beats the field if it matches FGSM-PCO's zero-collapse rate and comes within about one point of its AutoAttack accuracy at lower wall-clock time than the KL regularizer or FGSM-PCO. It also counts as a win if, stacked on either of those methods, it adds robustness or stability.
Why not just…?
Why not just add a clean-loss term with a tuned weight? This is the most direct threat, because the static-mix ablation uses the same free gradient. CA-AT reports that fixed convex combinations fail once $\cos(g_c,g_a)<0$, because any positive weighting then gives a compromise direction (Conflict-Aware Adversarial Training). That evidence comes from multi-step training, however. We sweep $\lambda$ with the same tuning budget as our method. If the best $\lambda$ removes collapse equally well, the projection adds nothing, and the contribution reduces to the diagnostic in stage one.
Why not just use an existing fix for catastrophic overfitting, such as the KL regularizer, SORA, or FGSM-PCO? These methods already work well, and FGSM-PCO's reported robustness exceeds PGD-10's. Our method must win on cost and on tuning sensitivity, or else complement them. We therefore report accuracy against wall-clock time and robustness across hyperparameter grids, and we test stacked combinations. If our method alone falls below them and adds nothing when stacked, it is not worth adopting.
Why not just apply PCGrad or the Harmonized Cone off the shelf? With two objectives, our update is PCGrad's projection. The novelty is not the projection but where the reference gradient comes from and the link to the mixed Hessian. HARMONIC's polyhedral computation changes nothing for two gradients, as shown above. It is used only in the three-objective case, where we test whether it outperforms PCGrad.
Why not just detect collapse and stop early? Early stopping with a PGD check is cheap, but it halts training rather than preventing the failure, and it caps robustness at the pre-collapse level. A cosine-triggered early-stopping arm tests the diagnostic's usefulness separately from the intervention.
Proposed Experiments
We use CIFAR-10 and CIFAR-100 with ResNet-18 and WideResNet34-10, Tiny-ImageNet, PathMNIST, and ViT-Small, under $\ell_\infty$ budgets of 8/255, 12/255, and 16/255. Evaluation uses AutoAttack through RobustBench, plus PGD-50, clean accuracy, collapse counts over 10 seeds, throughput, and peak memory.
Four checks decide the project. The first is whether the cosine signal leads collapse by more epochs than $\|H_{\theta x}\|_2$ or PertAlign, measured at a fraction of their cost. The second is whether $g_a-g_r$ tracks finite-difference Hessian-vector products. The third is whether the projection keeps the collapse rate at zero out of 10 seeds at 16/255 while the static mix fails. The fourth is whether the random-start reference gradient matches a true clean gradient to within noise.
A failure of the first check refutes the mechanism. A failure of the third check, while the static mix succeeds, reduces the work to a monitoring tool.
References
- Fast is better than free: Revisiting adversarial training
- A Training-Dynamics View of Catastrophic Overfitting: Understanding and Prevention
- Harmonized Cone for Feasible and Non-conflict Directions in Training Physics-Informed Neural Networks
- Conflict-Aware Adversarial Training
- Gradient Surgery for Multi-Task Learning
- SORA: Free Second-Order Attacks in Fast Adversarial Training
- Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspective
- RobustBench: a standardized adversarial robustness benchmark