Training Against the Attacks That Actually Transfer

A deployed image classifier that hides its weights is usually attacked by transfer: the attacker builds adversarial images on a model they own (a "surrogate") and sends them to the hidden model, hoping the fooling carries over. The standard defense is adversarial training, which retrains the model on its own worst-case perturbed inputs. That defense prepares for a stronger attacker than this threat model contains, one who can compute gradients through the deployed weights. The price is real. Each training step runs an inner attack loop that multiplies compute by roughly an order of magnitude, and clean accuracy drops by several points. We propose a cheaper target. Precompute, once, a bank of adversarial images from a varied pool of surrogate architectures (CNNs and Vision Transformers) that were each trained with only a small amount of adversarial robustness. Then train the deployed model on that bank with no inner attack loop at all.

The choice of "lightly robust" surrogates is the core of the idea, and it comes from attacker-side evidence. Why Does Little Robustness Help? shows that transferability is a bell curve in the surrogate's adversarial-training budget. Small budgets make the surrogate's gradients smooth while keeping them aligned with ordinary victim models. Large budgets push training onto unnatural, off-data-manifold inputs and break that alignment. So the perturbations an attacker would most want to use come from lightly robust surrogates. The same off-manifold mechanism is what TART blames for clean-accuracy loss in adversarial training. Our bet is that one choice serves both goals. A bank built from lightly robust, architecturally diverse surrogates should cover the most dangerous transfer directions. Because those directions lie closer to the data manifold, training on them should cost less clean accuracy than training on a model's own worst-case perturbations.

Core Claim

Against black-box transfer and surrogate-guided query attacks, training on a fixed bank from bell-curve-optimal surrogates matches or beats single-step adversarial training on transfer robustness. It does so at close to standard-training cost per target model and with a smaller clean-accuracy drop. The surrogate robustness level used to build the bank should matter in the same bell-shaped way it matters for attackers. We do not claim white-box robustness, and we will report white-box numbers so nobody mistakes the method for a white-box defense.

Written out, the target model $f_\theta$ is trained on

$$\min_\theta \; \mathbb{E}_{(x,y)}\Big[(1-\alpha)\,\mathcal{L}(f_\theta(x),y) + \alpha \max_{k \in S}\mathcal{L}\big(f_\theta(x+\delta_k(x)),y\big)\Big],$$

where $\delta_k(x)$ is the stored perturbation of image $x$ from surrogate $k$, $S$ is a small random subset of the surrogate pool drawn each step, and $\alpha$ is the mixing weight between clean and perturbed loss. The maximum over $S$ costs only $|S|$ extra forward passes. It needs no backward pass with respect to the input.

Relation to Prior Work

The obvious version of this idea already exists. Ensemble adversarial training adds transfer examples from fixed, standardly trained models. It appears in the file only as a baseline in Gray-Box Adversarial Training, whose authors report it is weak on their worst-case-over-checkpoints metric. Gray-Box Adversarial Training (GAT) instead attacks the model with its own earlier checkpoints, all of the same architecture, alongside white-box examples. Our proposal differs from both in three ways:

  1. Surrogates are chosen by the smoothness-versus-alignment principle rather than taken as standard models.
  2. They deliberately span architecture families.
  3. The target never runs an inner attack loop.

The ensemble-training comparison is the one most likely to sink the idea, so a dedicated ablation isolates the surrogate-robustness ingredient.

Several papers supply methods we reuse rather than motivate the idea:

  • Augmentation alignment. ATTA recycles perturbations across epochs of one model and maps them through random crops and flips with a "connection function." We use that function so stored perturbations stay aligned under on-the-fly augmentation. Unlike ATTA, our perturbations come from other models and are never updated by the target.
  • Fast training baseline. Fast is better than free replaces multi-step attacks with one randomly initialized step. It is our main cost competitor, but its robustness and clean accuracy were measured only white-box.
  • Robust teacher. IGAM transfers robustness by matching a robust teacher's input gradients without generating attacks. It needs a fully robust teacher, the costly artifact we avoid.

Two papers bear on the transfer-defense claim and constrain it. Q-TART reports improved resistance to transfer attacks by pruning noise-sensitive training samples, but it tested only standard source models. The Sword of Damocles in ViTs and GFCS show how strong current transfer and surrogate-guided query attacks are. They set the bar any transfer defense must clear.

Research Program

Surrogate pool. On CIFAR-10, CIFAR-100 and ImageNette, we train surrogates from four families: ResNet, VGG, a small ViT, and MobileNet. Each family is trained at three adversarial budgets (none, small, full), plus a SAM-with-input-gradient-regularization variant as in the "little robustness" paper.

Perturbation bank. For each surrogate we store PGD-10 perturbations (projected gradient descent, ten steps) at $\epsilon = 8/255$ for every training image, under a few augmentation states. We then train WideResNet and ViT targets on the bank.

Cost accounting. We count compute in two parts:

  • One-time cost: surrogate training plus bank construction.
  • Per-target cost: the training run itself.

The one-time cost is amortized over the many target models an organization trains, such as hyperparameter sweeps or periodic retrains. We report the break-even number of targets explicitly rather than hiding the one-time cost.

Clean-accuracy mechanism. To test the off-manifold explanation, we measure how far bank perturbations sit off the data manifold, using TART's tangent-space estimate on a subset. We correlate that distance with the clean-accuracy cost each surrogate's perturbations impose.

Baselines

Naive reference. Standard training with no defense. This gives the clean-accuracy ceiling and the transfer-robustness floor, at matched epochs and augmentation.

Fast adversarial training. FGSM with random initialization (FGSM is the fast gradient sign method, a one-step attack), per Fast is better than free. It reports 45% PGD robust accuracy on CIFAR-10 in 6 minutes; no clean accuracy is reported in the file. It is the strongest competitor on cost. We rerun it on the same architectures and epochs with the same tuning budget for step size and learning-rate schedule, keeping its PGD-validation early stopping. Beating it means equal or higher transfer-robust accuracy with at least 2 points higher clean accuracy, at lower per-target wall-clock.

Full multi-step adversarial training and TRADES. These are the strongest methods on robustness. TART reports standard adversarial training with WideResNet-32-10 on CIFAR-10 at 81.76% clean and 35.89% AutoAttack accuracy. TRADES reports 56.61% PGD robust accuracy. We run both on our setup. Beating them means matching their transfer-robust accuracy with higher clean accuracy at a fraction of the cost. No reported result was found on Lacuna for the transfer-attack accuracy of these models against SAM-regularized or ViT-redundancy surrogates, so we measure it ourselves.

Ensemble adversarial training with standard surrogates. This is our method with the bank drawn only from non-robust surrogates. It is the ablation that isolates the key ingredient.

Same-architecture recycling. GAT-style checkpoint perturbations, which GAT reports at 79.28% worst-case accuracy versus 56.11% for ensemble training on CIFAR-10 at $\epsilon=8/255$. We adapt it to our target with its inner white-box component removed. This separates the value of architectural diversity from that of using any other model at all.

Q-TART pruning. Evaluated against the same attack suite.

Why not just…?

Why not just use fast single-step adversarial training, which is already cheap and gives white-box robustness that should cover transfer too? This is the real threat. If fast training matches our transfer robustness and clean accuracy, the proposal collapses to "use fast training." Fast training still needs a forward-backward input-gradient pass every step and is known to collapse ("catastrophic overfitting") without careful tuning. Its clean-accuracy cost was never reported in the file. The head-to-head comparison above settles it. We commit to reporting a loss if one occurs.

Why not just do ensemble adversarial training with ordinary pretrained models? The "little robustness" result predicts that non-robust surrogates produce jagged, less-aligned gradients and therefore weaker, less representative transfer directions. GAT's numbers suggest ensemble training underperforms on worst-case transfer. The surrogate-budget ablation tests this directly. If standard surrogates do as well, the novelty reduces to architectural diversity and we will say so.

Why not just train the target itself with a small adversarial budget, or TRADES with a small trade-off weight? That likely recovers much of the clean accuracy, but it keeps the full inner-loop cost on every target. We include it at matched clean accuracy and compare the cost needed to reach equal transfer robustness.

Proposed Experiments

Attack suite and metric. The primary metric is worst-case accuracy over a fixed suite. On CIFAR-10, CIFAR-100 and ImageNette, the suite contains:

  • transfer attacks from SAM-with-input-gradient-regularization surrogates;
  • ViT-redundancy transfer attacks, which report an 86.9% average fooling rate on ImageNet;
  • GFCS query attacks, which report 99.9% success with a median of 6 queries on VGG-16 from ResNet-152.

All attacks are crafted from architecture families withheld from the bank, including DenseNet and Inception-style models. Withholding is the decisive test that the defense generalizes beyond its surrogates. Clean accuracy, per-target GPU-hours, amortized total GPU-hours, and white-box AutoAttack from RobustBench (as a disclosed diagnostic) are reported alongside.

Decisive checks. There are three:

  1. Bell curve: as the surrogate budget used for the bank varies, the target's transfer robustness should rise and then fall, mirroring the attacker-side curve.
  2. Held-out architectures: on withheld attacker families, the method should beat fast adversarial training on clean accuracy without losing transfer robustness.
  3. Bank-aware attacker: the attacker trains new lightly robust surrogates on the same data, which would expose overfitting to the specific bank.

If check one fails, the mechanism is wrong. If check two fails, fast training is the better practical answer.