Fixing the Weakest Class for Free: Flat Directions of Adversarial Training

Adversarial training makes an image classifier resistant to small, deliberately crafted input perturbations. It is known to leave some classes far more exposed than others. On CIFAR-10 a standard adversarially trained network reaches 67% robust accuracy on "automobile" but only 17% on "cat" (To be Robust or to be Fair), even though the training data are perfectly balanced. An attacker only needs the weakest class, so this gap matters. We propose to read this gap through parameter degeneracy, an idea from physics and cosmology. A degeneracy arises when many different parameter settings produce the same observation. Adversarial training "observes" only one number, the average robust loss. Many weight settings can share that number while protecting different classes. If part of the class gap sits along these directions where the average loss is flat, it can be reduced without losing any average robustness.

Concretely, we take an adversarially trained network and compute the few directions in weight space along which its average robust loss curves most sharply. We then fine-tune it to protect its weakest classes while forbidding movement along those sharp directions. This lets the model slide along the flat valley toward a setting that is equally good on average but fairer. The research question is how much of the class gap is "free" in this sense and how much is a real trade-off. We build a method to exploit the free part and a measurement that separates the two parts.

Core Claim

Let $R_c(\theta)$ be the robust loss of class $c$ at weights $\theta$ under a fixed attack, and let $R(\theta)=\frac{1}{C}\sum_c R_c(\theta)$ be the average over the $C$ classes. Standard adversarial training minimizes only $R$. Near a trained solution, $R$ is nearly flat along most directions of weight space, because the Hessian of $R$ has only a few large eigenvalues. Each class risk $R_c$ can still change a lot along those flat directions.

We claim that a measurable share of the class gap under adversarial training lives in this flat subspace. Moving along it with the update

$$\Delta\theta = -\eta\, P_k^{\perp}\, \nabla_\theta \Big( \tfrac{1}{\tau}\log \sum_c e^{\tau R_c(\theta)} \Big)$$

reduces worst-class robust error at near-zero cost in average robust and clean accuracy. In this update, $\eta$ is a step size. $P_k^{\perp}$ projects out the top-$k$ Hessian eigenvectors of $R$ and the gradient of $R$ itself. The log-sum-exp with temperature $\tau$ is a smooth maximum over class risks.

We use a smooth maximum over all classes, not a single hard worst class, for a specific reason. Enhancing Robust Fairness via Confusional Spectral Regularization showed that the class that is weakest during training is often not the weakest at test time.

The degeneracy view offers an explanation for that training–test divergence. If the weights are drifting along a flat valley, the identity of the worst class should fluctuate from step to step and from seed to seed while the average stays fixed. That is a testable prediction, not just a metaphor.

Relation to Prior Work

Three lines of work explain the class gap, and they motivate this proposal. On the Tradeoff Between Robustness and Fairness proves, for Gaussian mixtures, that adversarial training "sacrifices" hard classes, and it adds a class-variance penalty (FAT). Understanding the Impact of Adversarial Robustness on Accuracy Disparity attributes the gap to weight-norm shrinkage that shifts the boundary toward high-variance classes. Robustness May Be at Odds with Fairness traces weak classes to overlapping internal features, such as cat versus dog.

These papers also constrain the proposal. Their linear theory says fairness genuinely costs average accuracy. We therefore do not claim that all of the gap is free. We claim that deep networks add a degenerate component on top of that intrinsic cost, and we measure the split directly.

The existing fixes act on losses and outputs, never on weight-space geometry. FRL reweights the class losses and adapts a perturbation size per class (To be Robust or to be Fair). FAT penalizes the variance of class risks. CSR penalizes the largest singular value of a differentiable confusion matrix built from predictions (Enhancing Robust Fairness via Confusional Spectral Regularization). None of them asks which weight directions leave the average objective unchanged.

The method borrows its logic from cosmology.

  • Complementarity. Cosmic Complementarity and Weak Lensing: Prospects for Measuring Cosmological Parameters break degeneracies by adding a second probe whose Fisher information matrix (the curvature of the log-likelihood) points along the first probe's flat directions. Here, per-class risks play the role of separate probes that the average loss collapses together.
  • Decomposition. Cosmology with cosmic web environments showed that splitting one global statistic into environment-conditioned ones recovers information the global statistic hides.
  • Numerical care. Cosmic Complementarity also warns that one-sided finite-difference curvature estimates overstate precision. We therefore compute curvature only with exact Hessian–vector products averaged over several attack draws.

One last caution comes from Adversarial Robustness in Parameter-Space Classifiers. There, apparent robustness from a parameter-space construction turned out to be gradient obfuscation, and accuracy collapsed to about 9% on MNIST under adaptive attack. Our method changes only the weights of an ordinary network and adds nothing at inference. We nonetheless evaluate every model with AutoAttack.

Research Program

Step one: test the premise. We train ten seeds each of PGD adversarial training and TRADES on CIFAR-10 (ResNet-18 and WideResNet-34-10). We then ask whether, at matched average robust accuracy, the worst-class robust accuracy and the identity of the worst class vary across seeds and across late checkpoints. Large spread at fixed average is direct evidence of degeneracy. Near-zero spread would weaken the whole proposal early and cheaply.

Step two: build the geometric diagnostic.

  • For each trained model we estimate the top-$k$ Hessian eigenvectors of the average robust loss using Lanczos iteration on Hessian–vector products, with $k$ from 0 to 50.
  • For each class we compute its robust-loss gradient and split it into a part inside this sharp subspace and a part outside it.
  • The "degeneracy ratio" of a class is the fraction of its gradient norm lying outside the sharp subspace.

This is the direct analogue of asking which parameter combinations a probe constrains. We predict that weak classes, the confusable pairs identified by Robustness May Be at Odds with Fairness, have high degeneracy ratios.

Step three: exploit the flat directions. We apply projected smooth-worst-class fine-tuning to the trained models.

  • The eigenvectors are refreshed every few hundred steps.
  • Attacks are regenerated at every step, as in normal adversarial training.
  • The total budget is capped at 10% of the original training epochs.

We also measure the free-versus-costly split. We sweep $k$ and trace the curve of worst-class gain against average-robustness loss. The flat, near-zero-cost part of that curve is the degenerate component; the sloped part is the intrinsic trade-off that the Gaussian-mixture theory predicts.

Baselines

Reference points.

  • The naive reference is the adversarially trained model with no change.
  • A second reference continues ordinary adversarial training for the same extra 10% budget. This separates our effect from simply training longer.
  • The published anchors on CIFAR-10 are a worst-class robust error of 82.7% for standard PGD adversarial training (To be Robust or to be Fair) and a worst-class robust accuracy of 23.20% for TRADES (Enhancing Robust Fairness via Confusional Spectral Regularization).

Strongest competitors. CSR on top of TRADES is the strongest existing method in the file, reporting 36.30% worst-class robust accuracy on CIFAR-10 and raising few-shot-class robust accuracy on ImageNet-LT from 8.9% to 31.2% (Enhancing Robust Fairness via Confusional Spectral Regularization). FRL is the strongest reweighting-plus-margin method, cutting worst-class robust error from 82.7% to 69.5% at a cost of about 1–2% clean error (To be Robust or to be Fair). FAT is included as the variance-penalty representative; its only reported figure is roughly a 16% reduction in class-variance under C&W attack (On the Tradeoff Between Robustness and Fairness).

Fair comparison. We rerun CSR, FRL and FAT under the same protocol as our method:

  • the same architectures and data splits;
  • the same total epoch budget, counting our Hessian computations in wall-clock time;
  • the same hyperparameter search size, twenty configurations each, selected on a held-out validation split;
  • the same final AutoAttack evaluation.

Using one evaluation matters because RobustBench found that published defenses overstated robustness by sometimes more than 10% under weaker attacks.

What counts as winning. Beating CSR means higher worst-class AutoAttack accuracy at equal or better average robust accuracy. Alternatively, it means equal worst-class accuracy with a smaller drop in average and clean accuracy, which is the trade-off that every existing method pays.

Ablations.

  • The key ablation is identical fine-tuning with $k=0$ and no projection at all. It isolates whether restricting motion to flat directions is what makes the gain nearly free.
  • A second ablation removes only the average-gradient direction ($k=0$ but with projection). It tests whether curvature information adds anything beyond first-order gradient surgery.
  • A hard-maximum version instead of the smooth maximum tests the training–test divergence explanation.

Why not just…?

Why not just fine-tune with a worst-class-weighted loss and a small learning rate? This is the most serious threat, because a small step size also tends to keep the average loss roughly fixed. The evidence against it is indirect. Dynamic class weighting on ResNet-56 cut the worst-class deviation from −30% to −10%, but average accuracy fell from 78.9% to 75% (Robustness May Be at Odds with Fairness). Reweighting also chases a worst class that moves between training and test (Enhancing Robust Fairness via Confusional Spectral Regularization). The unprojected ablation, tuned over the same step-size grid, settles the question directly. If it traces the same worst-class versus average curve as the projected method, the geometric framing adds nothing and we will report that.

Why not just use CSR? CSR acts on predicted class confusions and already gives the best reported numbers. However, it is applied during training and still trades away average robustness to reach them. It also says nothing about how much of the gap was avoidable. Our method starts from an existing model, adds only a fine-tuning phase, and can be stacked on a CSR-trained model. Stacking tests whether flat-direction motion adds gains that CSR leaves unexploited. If stacking gives nothing, the degenerate component is already captured at the output level.

Why not just pick the fairest seed or checkpoint? If the class gap truly varies at fixed average, selecting among seeds is the cheapest way to exploit that variation. Step one measures exactly this and provides the selection baseline for free: the best-of-ten-seeds worst-class accuracy chosen on validation data. Our method must beat it, because it searches a continuous valley rather than ten sampled points. If seed selection matches our method, the practical recommendation becomes "train several seeds," which would still be a useful finding.

Why not just set a per-class perturbation budget, as FRL does? Per-class budgets change which examples are attacked, but they leave the weight-space geometry alone and still left worst-class error at 69.5% (To be Robust or to be Fair). FRL is run as a full baseline under the same budget, so the comparison decides this directly.

Proposed Experiments

Datasets and architectures.

  • The main study is CIFAR-10 with ResNet-18 and WideResNet-34-10, using PGD adversarial training and TRADES at $\epsilon=8/255$ under the $L_\infty$ norm.
  • CIFAR-100 tests whether the flat subspace still contains useful directions when there are many more classes. No reported result was found on Lacuna for robust-fairness methods on CIFAR-100.
  • ImageNet-LT, with its reported CSR numbers, tests the long-tailed regime at the scale where CSR's matrix decompositions become expensive.

Metrics. All metrics are computed under AutoAttack:

  • worst-class robust accuracy;
  • average robust accuracy;
  • clean accuracy;
  • the standard deviation of class robust accuracies;
  • training wall-clock time.

Decisive checks. The first check is step one's seed study. Substantial spread in worst-class accuracy at matched average accuracy must exist, or the proposal stops there.

The second check is the $k$-sweep curve. The proposal succeeds if some range of $k$ yields at least a few points of worst-class gain with under half a point of average-robustness loss. The unprojected ablation must fail to match it at equal tuning.

The third check is the degeneracy-ratio prediction: classes with high ratios before fine-tuning should be the ones that improve most. A null correlation here would mean the gains come from something other than the proposed mechanism, even if the headline numbers look good.