Small, carefully chosen input changes, called adversarial perturbations, can flip a neural network's prediction. Two facts about them are well established but rarely connected. First, a perturbation crafted on one model often fools a different model it was never computed on, which is called transferability. Second, training a model to resist such perturbations, which is called adversarial training, usually lowers its accuracy on ordinary unmodified inputs. This note proposes that both facts come from the same small set of input directions. These are the directions along which independently trained models agree that the loss rises fastest. If that is right, robustness along the "private" directions that only one model is sensitive to should cost almost no clean accuracy and buy no protection against transferred attacks. Robustness along the shared directions should carry most of the clean-accuracy cost and most of the protection against black-box attackers.
The idea extends a well-known linear picture. A tiny perturbation aligned with a weight vector shifts a linear model's output by $\epsilon\|w\|_1$, a quantity that grows with input dimension (Explaining and Harnessing Adversarial Examples). Our hypothesis concerns which part of that accumulation is common to models trained on the same data. The project builds a cheap, per-input estimate of the shared directions from a small pool of independently trained models. It then uses that estimate in two causal tests. The first projects attacks onto the shared directions, or away from them, and measures whether they still transfer. The second restricts adversarial training to one part or the other and measures which part causes the clean-accuracy loss.
Core Claim
Fix a pool of $K$ independently trained models (different seeds and architectures) and an input $x$. Let $S(x)$ be the span of the top $r$ principal directions of the pool's normalized input-loss gradients and of their attack perturbations at $x$. Let $P_S$ and $P_\perp$ be the projections onto $S(x)$ and its orthogonal complement.
For a perturbation $\delta$ crafted on a surrogate model, a first-order view of an unseen target model $t$ with logit-margin gradient $\nabla f_t(x)$ gives
$$\Delta_t \approx \nabla f_t(x)^\top P_S\,\delta \;+\; \nabla f_t(x)^\top P_\perp\,\delta .$$
The claim has two halves. On transfer, the second term is close to zero for targets independent of the surrogate, because private directions are uncorrelated across models. Transfer is therefore governed by $\|P_S\delta\|$. On the trade-off, clean accuracy relies on the same shared directions, so making a model insensitive along $S(x)$ is what pushes its decision boundary into the natural data.
The transfer half generalizes two motivating results from shallow settings. A Geometric Perspective on the Transferability of Adversarial Directions proves that the max-margin direction of a linearly separable dataset fools every linear separator of that data. Adversarial Vulnerability from Interference Between Features in Superposition shows that models trained on correlated data converge to the same feature-interference geometry, raising synthetic transfer from 5% to 98%. Neither work measures clean accuracy, evaluates defenses, or tests deep image classifiers end to end. We do all three.
The trade-off half refines the decomposition in Theoretically Principled Trade-off between Robustness and Accuracy, which splits robust error into natural error and boundary error. We ask in which input directions the boundary error that costs clean accuracy lives.
Research Program
The first component is the estimator. For CIFAR-10 we train $K=5$ standard models, two ResNet-18s, two WideResNets and one small Vision Transformer. For each test input we compute a few gradient and PGD (projected gradient descent, the standard iterative attack) perturbations per model. We then take an SVD of the stacked unit vectors to obtain $S(x)$ for $r$ between 1 and 32.
Two sanity checks anchor the estimator. On MNIST with linear SVMs, $S(x)$ should recover the max-margin direction of Charles et al. On a synthetic superposition bottleneck, it should recover the shared interference directions of the superposition paper.
The second component is the transfer test. We craft attacks on a surrogate that is not in the pool, then project each perturbation onto $S(x)$, onto $S(x)^\perp$, or onto a random subspace of equal dimension. Each projected perturbation is rescaled to the same $\ell_\infty$ and $\ell_2$ budget. We measure transfer to held-out targets, including convolutional-to-transformer pairs, where the survey A Survey on Transferability of Adversarial Examples Across Deep Neural Networks reports that transfer is weakest.
The third component is the trade-off test. We run adversarial training with the perturbation constrained to $S(x)$ (S-AT) or to $S(x)^\perp$ (⊥-AT), using equal budget and steps. We compare clean accuracy, white-box AutoAttack accuracy, and transfer-attack accuracy against full PGD-AT and TRADES. The prediction is that ⊥-AT keeps clean accuracy near that of standard training while leaving transfer robustness near zero, and that S-AT pays most of full AT's clean-accuracy cost while buying most of its transfer robustness.
Following On Evaluating Adversarial Robustness, we also attack each restricted model with an adaptive white-box adversary that knows the restriction. For example, it attacks S-AT models only along $S(x)^\perp$. This keeps us from mistaking a blind spot for robustness.
The fourth component turns the estimator into an evaluation tool for published defenses. A "shared-subspace transfer attack" crafts on the surrogate pool and keeps only the $P_S$ component. We apply it to TART and AR-AT, which report clean-accuracy gains but never measured transfer. We test whether those gains come with reduced robustness along $S(x)$, which is the region black-box attackers exploit.
Baselines
The naive reference point is a standard (non-adversarially trained) model under each attack. It fixes the floor for transfer rates and the ceiling for clean accuracy.
For the trade-off, full PGD-AT and TRADES are the unrestricted versions of our restricted training. They are the most direct controls, because S-AT and ⊥-AT differ from PGD-AT only in where the perturbation may point. All of these are run with identical architecture, epochs, budget, attack steps and hyperparameter search. TART reports 81.76% clean and 35.89% AutoAttack accuracy for standard AT on WideResNet-32-10 CIFAR-10. Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off reports 84.33% clean accuracy for TRADES.
The strongest existing method on the trade-off is AR-AT. That paper reports 90.89% clean and 50.77% AutoAttack accuracy on CIFAR-10 with WideResNet-34-10. It is the right competitor because it already narrows most of the clean-accuracy gap, so any directional explanation must also account for it. We reproduce it at the same architecture and budget.
TART: Boosting Clean Accuracy Through Tangent Direction Guided Adversarial Training is the closest method in spirit. It reports 83.73% clean and 36.58% AutoAttack accuracy for TART-AT on WideResNet-32-10. TART splits perturbations by an autoencoder-estimated data manifold, whereas we split by cross-model agreement. We do not aim to beat AR-AT's numbers with S-AT or ⊥-AT, which are diagnostic instruments.
Each hypothesis has its own success criterion. The trade-off claim succeeds if ⊥-AT recovers at least most of the clean-accuracy gap between PGD-AT and standard training, while S-AT retains most of PGD-AT's transfer robustness. The evaluation claim succeeds if the shared-subspace transfer attack lowers accuracy on AR-AT or TART more than the best standard transfer attacks at equal budget.
For transfer, the competitors are the established surrogate-side methods catalogued in the survey: momentum (MI-FGSM), input transformations, and backpropagation linearization (LinBP). No reported result was found on Lacuna for these methods on our exact surrogate–target pairs, so we run all of them ourselves with the same surrogates and query-free budget.
Three ablations isolate the key ingredient, which is agreement across models. The first replaces $S(x)$ with a random subspace of equal dimension. The second uses the top-$r$ gradient directions of a single model. The third uses TART's tangent subspace. The claim requires that the multi-model subspace beats all three on both the projection test and the restricted-training test. The pool's training cost is reported and charged to our method in every compute comparison.
Why not just…?
Why not just use TART's tangent space, since "on-manifold" and "shared" may be the same thing? This is the most serious threat. If the data manifold is what all models learn, the two subspaces could coincide. We measure principal angles between $S(x)$ and TART's tangent estimate on the same inputs, and run the projection and restricted-training tests with both. If they behave identically, the contribution shrinks to a cheaper, autoencoder-free estimator plus the transfer measurement TART never ran, and we will say so. If cross-model agreement predicts transfer and clean-accuracy cost better than tangency, the distinction is real.
Why not just measure gradient cosine similarity between surrogate and target, the usual transfer proxy? A scalar similarity can predict that transfer happens. It cannot remove the shared component from a perturbation or restrict training to it, so it cannot test whether those directions cause the clean-accuracy loss. The projection and restricted-training experiments are exactly what a similarity score cannot do.
Why not just do adversarial training against an ensemble of surrogates? Ensemble-based training would likely improve transfer robustness. It does not say where the clean-accuracy cost comes from, which is the question here. It also mixes shared and private directions in each perturbation. We include an ensemble-crafted-perturbation AT variant at matched compute. If it matches S-AT on both clean and transfer accuracy, the explicit subspace adds nothing for training. The diagnostic claim about where the cost is paid would still stand or fall on the ⊥-AT result.
Why not just eliminate superposition, which removed all adversarial examples in toy models? In the toy setting of Adversarial Vulnerability from Interference Between Features in Superposition, giving every feature its own dimension removed adversarial examples. Real image features far outnumber hidden dimensions, and the same paper finds that weight decay drives representations into more superposition, not less. Pruning shared channels damages clean accuracy, as backdoor work such as Adversarial Neuron Pruning Purifies Backdoored Deep Models shows. Our approach locates the shared directions in input space without requiring the representation to be rebuilt.
Decisive Experiments
The main runs use CIFAR-10 and CIFAR-100. Imagenette provides a higher-resolution check, and MNIST serves as the linear sanity case. Robustness is evaluated with AutoAttack (Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks) under the RobustBench: a standardized adversarial robustness benchmark protocol, at $\ell_\infty$ budget 8/255 and $\ell_2$ budget 0.5. Transfer targets are held-out seeds and architectures, plus RobustBench models.
The first decisive check is the projection test. At equal norm, $P_S\delta$ should transfer nearly as well as $\delta$, while $P_\perp\delta$ and random projections should transfer at near-floor rates. The transfer hypothesis is falsified if $P_\perp\delta$ retains a large share of transfer success for any $r \le 32$.
The second is the restricted-training test. The trade-off hypothesis is falsified if ⊥-AT costs as much clean accuracy as S-AT, or if neither restriction separates clean cost from transfer robustness. We also check that the adaptive attacker reduces each restricted model's white-box accuracy to the level its restriction predicts. We apply the gradient-masking diagnostics of Carlini et al.: iterative attacks must beat single-step ones, and accuracy must fall monotonically as the budget grows.
The third is the defense audit. We report AR-AT and TART accuracy under the shared-subspace transfer attack alongside MI-FGSM, input-transformation and LinBP baselines. If their clean-accuracy gains coincide with lowered robustness along $S(x)$, black-box robustness becomes a missing axis in current trade-off reporting.
We will also report how $r$, the subspace dimension needed to capture transfer, changes from MNIST to Imagenette. This tests whether shared vulnerability stays low-dimensional as input dimension grows, as the linear-accumulation picture predicts.
References
- Explaining and Harnessing Adversarial Examples
- A Geometric Perspective on the Transferability of Adversarial Directions
- Adversarial Vulnerability from Interference Between Features in Superposition
- Theoretically Principled Trade-off between Robustness and Accuracy
- A Survey on Transferability of Adversarial Examples Across Deep Neural Networks
- On Evaluating Adversarial Robustness
- Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off
- TART: Boosting Clean Accuracy Through Tangent Direction Guided Adversarial Training
- Adversarial Neuron Pruning Purifies Backdoored Deep Models
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
- RobustBench: a standardized adversarial robustness benchmark