Does Test-Time Noise Stop Transferred Attacks? Measuring the Coupling Between Transferability and Fragility

Adding small random noise to a model's input at inference time is one of the cheapest defenses against adversarial examples. These are inputs nudged by tiny, carefully computed changes so that a classifier gets them wrong. The noise works because many such changes push the input into a narrow "pocket" of wrong answers, and a random jolt knocks it back out. In the white-box setting, where the attacker has the defended model's gradients, this defense is known to be largely illusory. Repeated gradient steps average over the defense's randomness, and fine-tuning the model to tolerate the noise, which is needed to keep accuracy on clean inputs, removes whatever protection the noise gave. Most real attackers, however, never see the deployed model. They build the attack on their own substitute model and hope it carries over, which is called a transfer attack. Here nobody has checked whether test-time noise helps, or at what cost in clean accuracy.

We propose that, in the transfer setting, the two questions "does this perturbation transfer?" and "does it survive random noise?" are largely the same question. Both measure how wide the region of wrong answers around the perturbed input is. If so, test-time noise mostly removes perturbations that would not have transferred anyway, while the dangerous ones survive. We further predict a black-box version of the known Catch-22: fine-tuning the target to tolerate noise in order to recover clean accuracy makes its gradients more like those of a noise-smoothed substitute model. That would raise, not lower, the success of noise-aware transfer attacks. This note describes the measurements that would confirm or refute both claims.

Core Claim

Two predictions follow, and each can fail on its own. The first is a coupling prediction. For a perturbation built on a substitute model, we can measure on that same substitute the fraction of random Gaussian draws of scale $\sigma$ under which it stays adversarial. We call this its survival rate. We predict that the survival rate is a strong predictor of two things: whether the perturbation fools an unseen target, and whether it still fools that target when the target adds noise of scale $\sigma$. If this holds, a defender can estimate how much a noise defense will help against a given attack family using only substitute models, before deployment.

The second is an alignment prediction. Fine-tuning a target on noisy inputs restores its clean accuracy, but we predict it also raises the transfer success of attacks built with noise averaging. The mechanism is that both the attacker's objective and the fine-tuned target now approximate the same noise-smoothed classifier. The attacker's objective is the expected loss $\bar f(x+\delta)=\mathbb{E}_n[f(x+\delta+n)]$, where $x$ is the clean input, $\delta$ the perturbation, and $n$ the random noise. Under this prediction, the step that fixes clean accuracy is also the step that hands the attacker the target's geometry, with no gradient access through the defense required.

If the alignment prediction fails, meaning noise fine-tuning restores clean accuracy without raising transfer success, then there is a usable regime for cheap noise defenses against black-box attackers. That is a positive result worth having.

Motivating and Constraining Prior Work

Three papers motivate the gap. A New Defense Against Adversarial Images: Turning a Weakness into a Strength builds a detector on the observation that adversarial images fall back to their correct class under Gaussian noise while clean images do not. Its evaluation is white-box and gray-box, and it never asks how surrogate-crafted attacks behave. On the Limitations of Stochastic Pre-processing Defenses shows that iterative attacks break stochastic preprocessing through implicit averaging over its randomness. On ImageNette, fine-tuning a randomized-transform defense from about 20% to over 90% clean accuracy pushes attack success from near zero to nearly 100%. That paper explicitly leaves transferability unexamined, and our alignment prediction is the black-box analogue of its mechanism. Comment on Transferability and Input Transformation with Additive Noise argues theoretically that noise-averaged attack objectives generalize better across the space of classifiers. It runs no experiments and does not consider defender-side noise. We turn its argument into a measurable quantity, the survival rate, and test it against defended targets.

Two lines of work supply method. Benchmarking Transferable Adversarial Attacks (TAA-Bench) provides the attack families and the architectures. These include iterative, momentum-based, input-diversity, intermediate-feature and generative attacks, run across ResNets, Inception models and adversarially trained models. It does not include stochastic inference defenses. Two Sides of the Same Coin shows that fine-tuned models inherit transfer vulnerability from their source, and defines the transferability ratio $\gamma=(a_b-a_w)/a_w$, where $a_b$ is accuracy under the transferred (black-box) attack and $a_w$ is accuracy under a direct white-box attack. We reuse this ratio. Its findings rest on single-step attacks, so we rerun the question with iterative ones.

The purification literature constrains the design. Diffusion Models for Adversarial Purification and DiffAttack both report a "sweet spot" in noise level. Too little noise leaves the attack intact, and too much destroys the image content. We must therefore report every result as a curve over $\sigma$, not at a single tuned value.

Research Program

The program has three parts.

Survival-rate measurement. For each attack family in TAA-Bench, plus the noise-averaged attack of the Comment paper and a multi-substitute ensemble, we craft perturbations on a substitute. We then estimate the survival rate on that substitute over a grid of $\sigma$ values, using a fixed number of noise draws.

Targets. We deploy several variants of each target. The first is undefended. The second adds Gaussian noise at scale $\sigma$ with no retraining. The third adds the same noise after fine-tuning the target on noisy inputs, the clean-accuracy recovery step. For each target and attack we record clean accuracy, transfer success, and $\gamma$.

Analysis and ablation. The coupling prediction is tested by how well the substitute survival rate ranks per-example transfer success against each target variant. We report rank correlation and AUROC, and require that this beat simpler predictors such as perturbation norm or substitute loss. The alignment prediction is tested two ways. We compare transfer success against noise-fine-tuned and non-fine-tuned targets as clean accuracy is matched. We also measure the cosine similarity between the target's input gradient and the substitute's noise-averaged gradient before and after fine-tuning. An ablation isolates the mechanism by fine-tuning the target for the same budget on a different corruption, such as blur. This separates noise-specific alignment from generic effects of fine-tuning.

Baselines

Undefended target. The do-nothing reference is the target used as-is. It fixes the clean accuracy ceiling and the raw transfer success for each attack. Every defense is judged by how much transfer success it removes per point of clean accuracy lost.

Noise without fine-tuning. Inference-time Gaussian noise added to an unchanged target is the defense under study in its cheapest form. It is run over the full $\sigma$ grid so that it can be compared to other defenses at matched clean accuracy.

Diffusion purification. The strongest existing test-time competitor is DiffPure Diffusion Models for Adversarial Purification. It reports robust accuracy on CIFAR-10 that is 5.44 points above the best adversarially trained model under adaptive white-box attack. DiffAttack DiffAttack reduces a WideResNet-28-10 DiffPure pipeline from 70.64% to 46.88% at $\ell_\infty$, $\epsilon=8/255$. Score-based purification Adversarial Purification with Score-based Generative Models reports 70.01% under an adaptive attack combining gradient approximation (BPDA) with averaging over its randomness (EOT). For transfer attacks against any of these defenses: no reported result found on Lacuna. We run DiffPure with the same target classifiers and the same transferred perturbations, sweeping its diffusion timestep to match clean accuracy with the noise variants. It is the stronger defense, but it costs far more compute. We count cheap noise as competitive if, at matched clean accuracy, its transfer success is within a few points of DiffPure's. We count our survival-rate predictor as useful if it predicts DiffPure's residual failures as well as it predicts those of plain noise.

Adversarially trained target. This reference is a RobustBench model RobustBench. TART TART reports 83.73% clean accuracy and 36.58% AutoAttack accuracy for WideResNet-32-10 on CIFAR-10, against 81.76% and 35.89% for standard adversarial training. It is evaluated under the same transferred perturbations.

Deterministic preprocessing. JPEG compression and bit-depth reduction are run over their own strength grids. They are the cheap alternative a skeptic would reach for first.

Ablation. The blur-fine-tuned target described above isolates whether any rise in transfer success is specific to noise.

Why not just…?

Why not just use JPEG or bit-depth reduction instead of noise? APBench shows that these operations strip training-time poisons cheaply. IMPRESS finds that JPEG, random noise and resizing all fail against optimized protections. TAA-Bench includes frequency-domain attacks built to survive filtering. Deterministic preprocessing could still match noise against transfer attacks, so it is a real competitor. We include it on the same matched-clean-accuracy curves. If it dominates, we report that. The coupling analysis applies to it unchanged, with "survival under JPEG" replacing survival under noise.

Why not just adversarially train the target? Adversarial training does resist transfer, and TAA-Bench reports that transfer attacks weaken against hardened models. It costs clean accuracy, though: 81.76% clean accuracy in TART's standard-training baseline, versus models above 94% noted in APBench. It also requires retraining, which test-time noise avoids. Our question is whether the cheap, retraining-free regime is usable, so adversarial training serves as a reference frontier rather than the object of study.

Why not just assume the attacker knows $\sigma$ and averages over it, making the question trivial? The objection is partly right. Noise-averaged attacks should survive matched noise on the substitute. The open questions are whether survival on the substitute predicts survival on a different target, and whether the target's own fine-tuning increases transfer. Neither follows from the attacker's objective alone. The coupling experiment answers the first, and the gradient-alignment measurement with its blur ablation answers the second.

Proposed Experiments

We use CIFAR-10 with RobustBench and standard models, and ImageNet with the TAA-Bench architectures (ResNet-50/101/152, Inception-v3/v4), under $\ell_\infty$ budgets of 8/255 and 4/255 respectively. Each experiment holds out one architecture family as the target.

The coupling claim passes if the substitute survival rate predicts per-example success against noise-defended targets clearly better than perturbation norm or substitute loss do. It fails if the predictive power is near chance for noise-averaged and input-diversity attacks.

The alignment claim passes if, at matched clean accuracy, the noise-fine-tuned target has higher transfer success from noise-averaged attacks than the noise-only target, together with higher gradient cosine similarity to the substitute, and the blur-fine-tuned control shows neither effect. It fails if fine-tuning restores clean accuracy without raising transfer success. In that case we would report the $\sigma$ range where cheap noise offers real black-box protection, compared directly against DiffPure at matched clean accuracy.