Path-Integral Single-Step Adversarial Training

Making a neural network robust to small malicious input changes usually means training it against the worst perturbation found by an inner search. That search is the expensive part: each of its $K$ gradient steps costs a full forward and backward pass. The cheap shortcut is to take a single gradient step from a random start. It works until it suddenly does not. In "catastrophic overfitting," robustness to stronger attacks collapses to near zero within a few epochs. We propose to treat this inner search the way high-dimensional stochastic optimal control treats an intractable value function. Optimal control does not grid the space or iterate to convergence. It averages a few cheap simulated samples, weighted by how costly each one is, and centers the sampling on a good guess so the weights stay informative. Here the guess is the single gradient step. The samples are a handful of forward-only evaluations, with no backward pass, around that step. The weighting is an exponential "soft maximum" whose temperature, called viscosity in the control literature, is set automatically so the weights never collapse onto one sample.

The bet has two parts. First, a temperature-controlled soft maximum over a few gradient-centered samples prevents catastrophic overfitting at roughly the cost of one extra gradient step. It also uses less memory than current regularizers, because the extra samples need no backward pass. Second, a quantity borrowed directly from importance sampling in control gives a forward-only early warning of catastrophic overfitting. That quantity is the effective sample size of the weights, meaning how many samples meaningfully contribute. This lets us measure the curse of dimensionality in the attack itself: how the needed number of samples grows with input dimension, and whether centering on the gradient removes that growth.

Core Claim

For an example $x$ with label $y$, let $\delta_0$ be a uniform random start in the $\ell_\infty$ ball of radius $\epsilon$. Let $g$ be the input gradient at $x+\delta_0$, the one backward pass every single-step method already pays for. We draw $M$ candidate perturbations $\delta_j$ around the gradient step $\Pi(\delta_0+\alpha\,\mathrm{sign}(g))$, where $\Pi$ projects back into the ball. The candidates lie in a small subspace spanned by $\mathrm{sign}(g)$, the perturbation stored for this example in the previous epoch, and a few random sign directions. We evaluate the loss $L_j = \mathcal{L}(f_\theta(x+\delta_j),y)$ for each candidate with forward passes only, and form

$$w_j = \frac{\exp(L_j/\tau)}{\sum_{k=1}^{M}\exp(L_k/\tau)}, \qquad \delta^\star = \Pi\Big(\sum_{j=1}^{M} w_j\,\delta_j\Big),$$

where $\tau>0$ is the viscosity (temperature). The network then takes one ordinary training step on $x+\delta^\star$. The averaging step is the Cole-Hopf weighted-mean formula that A Hamilton-Jacobi-based Proximal Operator uses to compute proximal operators from samples. It is also the path-integral control rule used in Path integral formulation of stochastic optimal control with generalized costs, applied to a one-step "trajectory." The ingredient that makes it usable is how $\tau$ is chosen. Instead of hand-tuning it, we set $\tau$ for each batch by bisection so that the effective sample size

$$\mathrm{ESS} = \Big(\sum_j w_j^2\Big)^{-1}$$

hits a target fraction $\rho M$. This is the adaptive tempering used in sequential Monte Carlo.

When $\tau\to 0$ the rule picks the single worst candidate, a hard maximum. When $\tau\to\infty$ it returns the plain average, which amounts to noise augmentation. Fixing the effective sample size keeps the method between these failure modes no matter how jagged the loss surface becomes.

What is new is the combination of three pieces. The sampling is centered on the gradient step, in the role a Girsanov drift plays in control. The averaging is a viscous soft maximum. The temperature is controlled through the effective sample size. Each piece answers a known failure. Pure sampling suffers the curse of dimensionality: random noise in 3,072 dimensions rarely raises the loss. A pure hard maximum over a few checkpoints is fragile. A fixed temperature is one more sensitive hyperparameter, and that sensitivity is exactly what Fast is better than free: Revisiting adversarial training concedes about single-step training.

Relation to Prior Work

Several papers show the gap. Fast is better than free: Revisiting adversarial training shows that randomly started single-step training reaches PGD-level robustness in minutes but breaks under small changes in step size. Understanding Catastrophic Overfitting in Single-step Adversarial Training ties the collapse to distorted decision boundaries and input-gradient norms that grow up to 35×. Its fix is a hard search over three checkpoints along the gradient line ($\epsilon/3$, $2\epsilon/3$, $\epsilon$), which is our method's $\tau\to 0$, one-direction limit. We test whether softening that maximum and widening its support beyond the line is what matters.

Other work supplies methods we reuse. The weighted-mean estimator and the reading of viscosity as a smoothing knob come from A Hamilton-Jacobi-based Proximal Operator, which used it for black-box proximal steps up to 1,000 dimensions but never inside a min-max training loop. The control literature also explains why centering matters. Deep Forward-Backward SDEs for Min-max Control relies on Girsanov importance sampling, which shifts where samples are drawn without changing the problem. Stochastic Optimal Control Matching reports that importance weights blow up in variance in high dimensions when the sampling policy is far from optimal. We expect the same failure for uncentered sampling of perturbations and plan to measure it.

The Hamilton-Jacobi Theory of Deep Learning argues that a viscosity parameter governs Hessian bounds and robustness. Its exact correspondence holds only for log-sum-exp layers with quadratic Hamiltonians. We therefore use it as motivation, not as a guarantee for ReLU networks. Our temperature acts on the inner objective, not on the architecture.

Other work constrains the design. You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle already brought Pontryagin's maximum principle to adversarial training. It decouples the adversary from all but the first layer, and its speedup shrinks when that layer is heavy. Our extra cost is whole-network forward passes run in inference mode, so it does not depend on the stem and is equally cheap on convolutional and transformer stems. Efficient local linearity regularization to overcome catastrophic overfitting (ELLE) prevents collapse by penalizing non-linearity along interpolated points. Its extra forward passes must be differentiated, which raises GPU memory. Ours are not differentiated.

Research Program

The first strand builds the method and makes it cheap. All $M$ candidate evaluations run as one batched forward pass with gradients disabled and in mixed precision. Their activations are discarded, so peak memory should stay close to that of plain single-step training. The temperature bisection operates on $M$ scalars per example and costs almost nothing. Storing the previous epoch's perturbation for each example plays the role of the warm start in Adversarial training for free! without its repeated minibatches. We count cost in forward-pass equivalents, taking a backward pass as about two forwards. Under that accounting, $M=3$ matches the cost of two-step PGD.

The second strand is the curse-of-dimensionality analysis that gives the control framing its content. At a fixed $M$, we compare three proposal distributions. The first is isotropic random noise in the full input space. The second is the gradient-centered subspace. The third is gradient-centered sampling in a random subspace of the same size. For each, we track how much the chosen $\delta^\star$ raises the loss compared with a 50-step PGD reference, and how the effective sample size behaves at a fixed temperature. We do this across input dimensions from CIFAR-10 (3,072), Tiny ImageNet (12,288), and downsampled versus full-resolution ImageNet. The prediction from importance sampling is concrete. Isotropic proposals should become useless as dimension grows: the weights go flat, and the loss gain shrinks toward that of random noise. Gradient-centered proposals should keep a roughly dimension-independent gain.

The third strand is diagnostic. In deliberately unstable Fast AT runs (large $\epsilon$, large step size), we log the effective sample size at a fixed temperature for a held-out probe batch, next to the gradient-norm signal from the catastrophic overfitting paper. We then ask whether a drop in effective sample size precedes the collapse in PGD-50 accuracy, and by how many iterations.

Baselines

The naive reference point is Fast AT, from Fast is better than free: Revisiting adversarial training: one random start, one sign-gradient step, cyclic learning rate, and mixed precision. Its reported result is 45% PGD robust accuracy on CIFAR-10 in 6 minutes. It is the cost floor, and we run it with the same architecture (PreAct ResNet-18), schedule, and $\epsilon$ grid as everything else, including settings where it is known to collapse.

The strongest existing method against catastrophic overfitting in the file is ELLE-A from Efficient local linearity regularization to overcome catastrophic overfitting. It reports 44.32% AutoAttack accuracy on CIFAR-10. We will rerun it from the official code, LIONS-EPFL/ELLE, with the same architecture, epochs, data augmentation, and an equal hyperparameter search budget of the same number of configurations per method. We then evaluate both with AutoAttack through RobustBench: a standardized adversarial robustness benchmark. Beating it means equal or higher AutoAttack accuracy at $\epsilon=8/255$ and strictly higher accuracy at $\epsilon=16/255$, with no collapse in any of five seeds, at lower peak memory and no more wall-clock time.

The closest mechanism is the adaptive-magnitude checkpoint method from Understanding Catastrophic Overfitting in Single-step Adversarial Training. It reports 33.9% PGD-50 robustness in 3.9 hours, against 11.1 hours for PGD-7. We rerun it in our setup rather than compare numbers across papers.

Two compute-matched references matter most for honesty. The first is two-step PGD with a random start, at the same forward-pass-equivalent budget as our $M=3$ setting. The second is plain PGD-10 as the robustness ceiling. Free AT and YOPO, with reported figures of about 46% PGD-20 in 13 hours and 47.98% PGD-20 on Wide ResNet-34, are included for context on CIFAR-10. Their numbers come from different architectures, so they are not head-to-head comparisons.

Three ablations isolate the key ingredient. The first is a hard maximum over the same $M$ candidates ($\tau\to 0$). The second is a fixed hand-tuned $\tau$ with no effective-sample-size control. The third keeps the full method but samples isotropically instead of around the gradient.

Why not just…?

Why not just take the hard maximum over more checkpoints, as the catastrophic overfitting paper does? This is the most direct threat, since our method contains it as the $\tau\to 0$ limit. Our argument is that a hard maximum over a few samples is itself a jagged function of the network weights: the chosen perturbation jumps between candidates as the landscape distorts. A soft maximum with controlled effective sample size changes smoothly. The hard-maximum ablation at the same $M$ settles this. If it matches the full method at 16/255 across seeds, the temperature contributes nothing, and the contribution reduces to wider sampling plus the diagnostic.

Why not just spend the extra compute on a second gradient step? Two-step PGD at matched cost is a serious competitor and may well be as robust. Our claim is narrower and testable. Forward-only samples should match its robustness with lower peak memory and better parallelism, because the candidates run in one batch while gradient steps are sequential. They also yield the effective-sample-size signal, which two-step PGD does not. If two-step PGD wins on AutoAttack at matched wall-clock time, we will say so and present the method only as a memory-light option.

Why not just use ELLE or another local-linearity regularizer? ELLE already prevents collapse and reports 44.32% AutoAttack accuracy. Its stated costs are extra forward passes that must be differentiated and higher memory. Ours are not differentiated. The comparison is therefore on peak memory, wall-clock time, and accuracy at large $\epsilon$, where regularizers need retuning. Combining the two is a natural follow-up, not a reason to skip the test.

Why not just add random noise to inputs? That is our $\tau\to\infty$ limit with isotropic sampling. The dimension-scaling experiment is designed to show it fails in high dimensions, because random directions in thousands of dimensions barely raise the loss.

Proposed Experiments

The main study runs on CIFAR-10 and CIFAR-100 with PreAct ResNet-18 at $\epsilon\in\{8,12,16\}/255$, with five seeds each, reporting clean accuracy, PGD-50 accuracy, AutoAttack accuracy, wall-clock time, peak GPU memory, and the fraction of runs that collapse. SVHN and Tiny ImageNet test transfer. An ImageNet run with ResNet-50 at $\epsilon=4/255$ tests scale against Fast AT, which reports 43% robust accuracy in 12 hours. A ViT-S run tests the claim that, unlike YOPO, the method is indifferent to how heavy the first layer is.

Three checks decide the outcome. First, the full method must avoid collapse in every seed at 16/255 while Fast AT collapses. It must also beat the hard-maximum ablation, which shows that viscosity is doing work, and match or beat compute-matched two-step PGD on AutoAttack. Second, in the dimension sweep, isotropic proposals should lose most of their loss gain from CIFAR-10 to full-resolution ImageNet, while gradient-centered ones should not. That is the curse of dimensionality and its control-theoretic remedy, observed directly in the attack. Third, in unstable Fast AT runs, the drop in effective sample size should precede the PGD-50 collapse more reliably than the gradient-norm spike. If the first check fails but the third holds, the work still delivers a cheap, forward-only early warning for catastrophic overfitting.