Adversarial training makes a classifier robust by training it on deliberately corrupted inputs. It usually costs clean accuracy, because the model is forced to ignore every small change to the input, including changes that carry useful signal. A newer variant, latent adversarial training, corrupts the network's hidden activations instead of its pixels. Surprisingly, it has been reported to raise both robustness and clean accuracy. We propose to restrict those hidden-layer corruptions to the directions a model shares with an independently trained peer network, and to leave alone the directions that are specific to one model. The idea comes from work on adversarial transferability, the fact that an attack built on one model often fools another. That work suggests the directions shared across models are the ones attackers actually exploit. Directions specific to a single model are quirks of training. Defending them may spend capacity and cost clean accuracy for little real security.
In two sentences: we fit a cheap linear map between the hidden layers of two independently trained networks to find the subspace they agree on, then run latent adversarial training only inside that subspace. We test whether this keeps the robustness gains, especially against black-box transfer attacks, while removing the remaining clean-accuracy cost. This gives a data-driven, inside-the-network analogue of training only on "on-manifold" perturbations.
Core Claim
Harnessing the Vulnerability of Latent Layers in Adversarially Trained Models shows that input-level adversarial training leaves inner layers fragile. It then fine-tunes the network so later layers tolerate worst-case perturbations of earlier activations, reporting CIFAR-10 PGD accuracy rising from 47.12% to 53.84% and clean accuracy from 86.8% to 87.80%. Its perturbations fill an unconstrained ball in activation space, and the layer to perturb is chosen by trial and error.
TART supplies the motivating diagnosis. Perturbations that leave the data manifold force convoluted decision boundaries and drive clean-accuracy loss. TART fixes this with an external autoencoder that estimates tangent directions, but only in pixel space.
Adversarial Vulnerability from Interference Between Features in Superposition supplies the selection principle. Attacks work by amplifying overlapping feature directions. Models trained on correlated data converge to the same overlap geometry, which is why transfer rates in its synthetic setting rise from 5% to 98%.
Our claim combines these three results. The latent directions worth hardening are those shared across independently trained models, and the clean-accuracy cost of latent adversarial training is concentrated in the model-specific remainder. This claim is falsifiable. If it is wrong, training only on the shared component will lose robustness, or training only on the specific component will be as cheap in clean accuracy as training on the shared one.
Research Program
We first train a peer network on the same data with a different seed, and in a second condition with a different architecture. The main model's activations at layer $l$ are $h_l(x)$, and the peer's activations at a matched layer are $h'_l(x)$. On clean training data we fit a ridge-regularized linear map from $h_l$ to $h'_l$. We take the top $r$ canonical directions of the pair as the shared subspace, with orthogonal projector $P_s$. Here $r$ is set by a canonical-correlation threshold, not tuned per run.
The training objective follows the latent adversarial training recipe, but the latent perturbation is projected:
$$\min_\theta \; \mathbb{E}_{(x,y)}\Big[ \max_{\|\delta_x\|_\infty \le \epsilon} L(f_\theta(x+\delta_x),y) \;+\; \lambda \max_{\|\delta\|_2 \le \epsilon_l} L\big(g_l(h_l(x) + P_s\delta),y\big)\Big]$$
In this objective, $f_\theta$ is the full network and $g_l$ is the sub-network after layer $l$. $\epsilon$ and $\epsilon_l$ are the input and latent budgets, and $\lambda$ weights the latent term. Projected gradient ascent applies $P_s$ at every step. Because the main model drifts during training, we refit the map every few epochs. This is cheap: it is a single regression on cached activations.
The program has three parts. The first is the method itself. The second is a layer-selection rule: we perturb the layer where the shared subspace is largest relative to layer width. This tests whether cross-model agreement replaces the ad hoc layer search reported for latent adversarial training. The third is a mechanistic check. We use the sparse-autoencoder superposition measure of Superposition as Lossy Compression, an entropy-based count of effective features per neuron, to ask how each variant changes feature packing. This bears on the open tension between the information-discarding account of Adversarial Training Reduces Information and Improves Transferability and the interference account.
Our use of a second model differs from TriQDef. TriQDef penalizes feature similarity across bit-widths of one quantized backbone to block patch transfer, so it pushes models apart. We use cross-model agreement only to decide where to train robustness, and we never alter the peer.
Baselines
The naive reference points are standard training and plain input-space adversarial training of the same architecture with the same schedule. The latter is the starting checkpoint for every latent variant, so every gain is measured from it.
The most direct competitor is unconstrained latent adversarial training as specified by Kumari et al. (paper). It is the method we modify, and it already reports simultaneous clean and robust gains (87.80% clean, 53.84% PGD on CIFAR-10). We run it with the same layer, latent budget, epochs and $\lambda$-tuning grid as our variant. "Beating it" means equal or higher AutoAttack accuracy together with a clean-accuracy gain larger than seed-to-seed variation across three seeds.
The strongest published trade-off result in the file is ARREST (paper). It fine-tunes a clean pre-trained model adversarially while distilling its hidden representations toward the frozen clean model, and reports 90.24% clean and 50.20% AutoAttack accuracy on CIFAR-10. Its recipe depends on a clean pre-trained model, so we give it the same clean checkpoint and the same fine-tuning budget. We also report our method stacked on ARREST, since the two are compatible. Beating ARREST on its own terms means a better position on its ARDist metric, which measures distance from an ideal trade-off curve, at matched compute.
TART (paper) is the input-space manifold competitor; it reports 83.73% clean and 36.58% AutoAttack against 81.76% and 35.89% for its adversarial-training baseline. We rerun it in our architecture rather than comparing against that number directly, because its baseline accuracy differs from Kumari's.
Three ablations isolate the key ingredient. The specific-only variant projects onto the complement $I-P_s$. The random-subspace variant uses a random projector of the same rank. The variance variant uses the top-$r$ principal components of the model's own clean activations. All three are run at matched latent perturbation norm. Without this control, a gain from projecting could simply reflect a smaller effective budget.
For black-box robustness, we attack with the strongest transfer attacks in the TransferAttack suite. Devling into Adversarial Transferability reports 86.4% average untargeted success for the Distribution Relevant Attack (DRA) and 74.1% for the More Bayesian Attack (MBA) on its ImageNet-compatible 1,000-image set. We found no reported result on Lacuna for these attacks against latent-trained models or on CIFAR-scale targets, so we establish those numbers ourselves.
Why not just…?
Why not just do input-space adversarial training against examples crafted on the peer model? This is a serious threat, because it uses the same extra model with less machinery, and the file contains no evidence against it. We run it at the same peer-training and attack cost. Our claim predicts it will improve black-box robustness but not clean accuracy, since it still trains on full input balls, which TART identifies as the source of clean-accuracy loss. If it matches our method on both axes, the latent projection adds nothing and we will report that.
Why not just project latent perturbations onto the model's own top principal components, with no peer at all? This is the cheapest version of the manifold idea inside the network, and it may work as well. The variance ablation at matched rank and norm settles it. Our hypothesis requires shared directions to differ from high-variance ones. The superposition account suggests attacks exploit overlapping, not necessarily dominant, directions (paper), but this is untested. If the principal-component variant ties, we will present the simpler method and treat the peer only as a diagnostic for why it works.
Why not just inject random noise into hidden layers? Kumari et al. compared latent adversarial training with feature-noise training and found noise inferior on every benchmark they tested (paper). We keep a noise run only as a sanity floor. The random-subspace ablation is the stronger version of this objection, because it keeps worst-case search but removes direction selection.
Why not just add unlabeled data? RobustBench identifies pseudo-labeled extra data as the most reliable way to raise both accuracies (paper), and ARREST reaches 93.17% clean accuracy with it. This route is complementary rather than competing. It changes the data, while we change where robustness is enforced. We report results without extra data for clean attribution, and include one run with extra data to check that the gains add.
Proposed Experiments
We run CIFAR-10 and CIFAR-100 with a ResNet-18 and a WideResNet main model, using the standard $\ell_\infty$ budget of 8/255, then scale to Restricted ImageNet. Peers are a second seed of the same architecture and, separately, a VGG or small Vision Transformer. White-box robustness is measured with full AutoAttack through RobustBench's protocol, because that benchmark documents overestimates of more than 10% under weaker attacks. Internal robustness is measured with Kumari's Latent Attack.
For transfer, the main model serves both as a target and as a source. It is a target for TransferAttack methods crafted on a pool of third-party surrogates, and the pool must exclude the peer used in training; otherwise the experiment is circular. It is a source when we test whether attacks crafted on it reach held-out CNNs and Vision Transformers. The second test matters because Two Sides of the Same Coin and Publishing Efficient On-device Models Increases Adversarial Vulnerability show that released models become attack surrogates for related private models.
The first decisive check is the shared-versus-specific split at matched norm. Our claim predicts that shared-only keeps most of the AutoAttack gain of unconstrained latent training and all of its black-box gain, with higher clean accuracy. It also predicts that specific-only costs clean accuracy while giving little black-box benefit. The second check is held-out transfer robustness: shared-only should resist DRA and MBA from surrogates it never saw, including other architecture families. The third check is layer selection: the layer with the largest shared fraction should match the best layer found by exhaustive search across depths.
A clear failure mode is that white-box AutoAttack accuracy needs the specific directions while transfer robustness does not. That outcome would itself be a finding, because it would separate the two kinds of robustness by representation direction. The superposition measurements then tell us whether shared-only training preserves more effective features than unconstrained training. If so, that would explain the accuracy gain through interference rather than information discarding.
References
- Harnessing the Vulnerability of Latent Layers in Adversarially Trained Models
- TART: Boosting Clean Accuracy Through Tangent Direction Guided Adversarial Training
- Adversarial Vulnerability from Interference Between Features in Superposition
- Superposition as Lossy Compression — Measure with Sparse Autoencoders and Connect to Adversarial Vulnerability
- Adversarial Training Reduces Information and Improves Transferability
- TriQDef: Disrupting Semantic and Gradient Alignment to Prevent Adversarial Patch Transferability in Quantized Neural Networks
- Adversarial Finetuning with Latent Representation Constraint to Mitigate Accuracy-Robustness Tradeoff
- Devling into Adversarial Transferability on Image Classification: Review, Benchmark, and Evaluation
- RobustBench: a standardized adversarial robustness benchmark
- Two Sides of the Same Coin: White-box and Black-box Attacks for Transfer Learning
- Publishing Efficient On-device Models Increases Adversarial Vulnerability