Natural images occupy a thin, low-dimensional surface inside pixel space. A 32×32 color image has 3,072 numbers, but the directions along which real images actually vary are far fewer. Prior theory shows that a network trained on such data leaves its weights largely unconstrained in the leftover directions, so tiny perturbations along them can flip predictions. We propose that this same "off-manifold void" is where backdoors hide. A backdoor is a hidden rule planted by poisoning a few training images with a trigger pattern so that any input carrying the trigger gets an attacker-chosen label. Our central claim is that whole-image backdoor triggers, such as blended overlays, sinusoidal color stripes and subtle warps, succeed mainly through their off-manifold component. If that is true, the size of the gap between pixel dimension and data dimension should predict how cheaply a model can be backdoored. It should also predict whether a trigger's effect carries over to other models.
The claim has two consequences. First, it gives a single geometric account of three disagreements in the literature: whether adversarial perturbations activate or erase backdoors, why backdoors transfer across class pairs inside one model but not across models, and why weak adversarial training can make a backdoor stronger. Second, it suggests a cheap defense. Adversarial training is the expensive procedure of training on worst-case perturbed inputs. The best current way to use it against backdoors runs several perturbation searches for every batch. We would replace that with a single-step perturbation confined to the off-manifold directions. Whether that defense beats simpler options is an empirical question that we set up to settle honestly. The geometric account is the main contribution either way.
Core Claim
Let $D$ be the input (ambient) dimension, $d$ the intrinsic dimension of the data, and $g = D - d$ the gap between them. Let $P_\perp$ be the projector onto an estimated off-manifold subspace, which is the complement of the top-$k$ principal directions of the training images or of a local tangent estimate. For any input-space vector $v$ (a trigger, an adversarial perturbation, or a difference of images), define its off-manifold fraction
$$\rho(v) = \frac{\|P_\perp v\|_2^2}{\|v\|_2^2}.$$
Here $\rho(v)$ is the share of the vector's energy that lies outside the data's principal directions. We hypothesize three things:
- Implantation scales with the gap. For whole-image triggers, the smallest trigger norm that achieves high attack success falls as $g$ grows while $d$ is held fixed.
- The backdoor shortcut is off-manifold. Untargeted adversarial perturbations that land in the backdoor target class have high $\rho$, and removing their off-manifold component removes that landing.
- Only the on-manifold part transfers. What makes a perturbation transfer to an independently trained model is its on-manifold component. The off-manifold component is specific to one network's unconstrained weights.
The motivation comes from Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerability. That paper decomposes two-layer ReLU weights into "authentic" on-manifold and "volatile" off-manifold parts. It proves that the perturbation budget needed for misclassification shrinks with $g$, and it shows that padding MNIST or upsampling ImageNet raises vulnerability while clean accuracy is unchanged. It studies test-time attacks only. We carry the same decomposition over to training-time poisoning.
We do not assume a specific exponent for the backdoor scaling. We fit the observed relation between trigger norm and $g$ and compare its shape with the paper's Theorem 4.2. The dimension-dependent shrinking of adversarial distance shown by Adversarial Spheres and Adversarial vulnerability for any classifier supplies the reason to expect that some such law exists. The observation in Adversarial Examples in Multi-Layer Random ReLU Networks that this sensitivity exists even before training suggests the void is a property of the architecture and the data, not of a particular optimizer.
What the Claim Explains
Two papers seem to disagree. Progressive Backdoor Erasing via connecting Backdoor and Adversarial Attacks (PBE) reports that untargeted adversarial examples on a backdoored model fall into the target class. They also sit close to triggered images in feature space, at a feature distance of 22.74 versus 148.26 in benign models. Adversarial-Inspired Backdoor Defense via Bridging Backdoor and Adversarial Attacks (AIBD) reports that perturbing backdoored samples flips them back to their true labels.
On a close reading, the two papers perturb different starting points. PBE perturbs clean images, and AIBD perturbs images that already carry the trigger. The geometric account predicts both outcomes. From a clean image, the cheapest way out of the true class is the unconstrained off-manifold direction the trigger carved, so the attack lands in the target class. From a triggered image, the cheapest way out of the target class is to cancel that same off-manifold component, which returns the image to its true label. We will test this directly. We will measure $\rho$ of the perturbations in both setups and check whether projecting out $P_\perp$ reverses each outcome.
A second tension concerns transferability, meaning whether an attack crafted on one model also fools others. NTD: Non-Transferability Enabled Backdoor Detection detects triggers because they fail to fool an independent clean feature extractor. UMD: Unsupervised Model Detection for X2X Backdoor Attacks detects backdoors because reverse-engineered triggers transfer across class pairs within one model. The volatile-weight picture reconciles them. The off-manifold weights are shared across classes inside one network, but they are effectively random across networks. That predicts transfer across class pairs within a model and failure of transfer across models.
A third observation comes from On the Effectiveness of Adversarial Training against Backdoor Attacks. It found that raising the $\ell_\infty$ adversarial-training budget first increases attack success and only later decreases it. Our account predicts the peak should occur where the training perturbation's off-manifold energy is still below the trigger's off-manifold norm. At that point the model is pushed toward the trigger direction without the direction being filled.
Research Program
The work proceeds in four stages, all built on the attack implementations and evaluation protocol of BackdoorBench: A Comprehensive Benchmark of Backdoor Learning.
Stage 1: scaling. We vary $D$ while holding $d$ fixed, using zero-padding and upsampling of MNIST and CIFAR-10 in the same way as the dimension-gap paper. MNIST's intrinsic dimension is estimated there at roughly 13–15 against an ambient 784. We implant Blended, SIG, WaNet and BadNets triggers. For each $g$ we find the smallest blend ratio or trigger norm that reaches 90% attack success. Patch triggers are the control case. They are localized and arguably more on-manifold, so the theory predicts they scale weakly with $g$.
Stage 2: decomposition. On backdoored PreAct-ResNet18 models, we compute $\rho$ for three kinds of vectors: triggers, PBE-style adversarial perturbations of clean images, and AIBD-style perturbations of triggered images. We then rerun each attack with its perturbation restricted to the on-manifold subspace or to the off-manifold subspace. We also sweep the adversarial-training budget to test where the non-monotonic peak falls.
Stage 3: transfer. We craft perturbations on a backdoored model and split each into its on-manifold and off-manifold components. We measure how often each component fools independently trained clean models: a different-seed ResNet18, an EfficientNet-B3, and a ViT. This also tests Gap 3 from ordinary transferability work, namely whether transferred perturbations travel along the manifold or through the shared void.
Stage 4: defense. We train with off-manifold single-step adversarial training, called OM-AT. Each step uses the perturbation
$$\delta = \epsilon \, \frac{P_\perp \nabla_x \mathcal{L}}{\|P_\perp \nabla_x \mathcal{L}\|_2}.$$
Here $\nabla_x \mathcal{L}$ is the gradient of the training loss with respect to the input, and $\epsilon$ is the budget. Optionally, we warm-start $\delta$ from the previous epoch's perturbation, following Efficient Adversarial Training With Transferable Adversarial Examples. The cost is about twice that of plain training plus one multiplication by a $D \times k$ basis.
Manifold estimation is a known weak point. We therefore report every result at several values of $k$ and with two estimators, global principal components and an autoencoder tangent. A conclusion counts only if it holds under both.
Baselines
The naive reference is undefended training on the poisoned data. The second naive reference is the same full-space single-step adversarial training as OM-AT with the projection $P_\perp$ removed. This is the key ablation, because it isolates whether confining the perturbation off-manifold is what does the work.
The strongest competitor on cost and coverage is Composite Adversarial Training (CAT), from On the Effectiveness of Adversarial Training against Backdoor Attacks. CAT combines $\ell_\infty$ and spatial adversarial training. It reduces the attack success rate (ASR) of all tested attacks to below 6% on CIFAR-10, and its spatial component reaches about 85% clean accuracy with near-0% ASR against patch attacks. No reported wall-clock cost for CAT was found on Lacuna, so we will measure it ourselves on identical hardware.
The strongest competitors on post-hoc erasure are PBE and AIBD. PBE reports about 1% ASR with about 94% clean accuracy on CIFAR-10. AIBD reports average ASR reduced from 99.73% to 0.33% on CIFAR-10. Both run iterative multi-step adversarial attacks, so their GPU-hours are part of the comparison.
We also include Subspace Adversarial Training (Sub-AT), which restricts weight updates to an 80-dimensional subspace of parameter space. Sub-AT reports over 51% robust accuracy against 50-step attacks at about a 7× speedup over 10-step training. It has never been tested on backdoors. ATTA-style recycled $\ell_\infty$ training is a further baseline.
Every method gets the same poisoned data, the same architecture, the same epoch budget and an equal hyperparameter search. The search covers 12 configurations, and the budget is selected on a held-out poisoned validation split. We report clean accuracy, ASR, recovered accuracy on triggered inputs (R-Acc), and GPU-hours. OM-AT beats the field if it matches CAT's sub-6% ASR on whole-image triggers within 2 points of CAT's clean accuracy at a fraction of CAT's measured cost. We will also report, and expect, that it does not match spatial training on patch triggers.
Why not just…?
Why not just run CAT with fast single-step inner loops? This could work, and we include it as a baseline. Making each $\ell_\infty$ and spatial step single-step still spends perturbation energy on on-manifold directions that the theory says are irrelevant to whole-image triggers. Single-step training is also prone to catastrophic overfitting, a sudden collapse of robustness. If fast CAT matches OM-AT at equal cost, the defensive half of this proposal adds nothing. The geometric findings of Stages 1–3 would still stand.
Why not just use Sub-AT, which already makes single-step training stable? Sub-AT builds its parameter subspace from early training checkpoints. On poisoned data, those early checkpoints may already encode the backdoor, so constraining updates to that subspace could preserve the backdoor rather than remove it. The prediction is easy to test, because Stage 4 trains Sub-AT on poisoned data and measures ASR.
Why not just add random noise in the off-manifold subspace, or project test inputs onto the manifold? The random-to-adversarial gap from Robustness of classifiers: from adversarial to random noise says random noise in a $g$-dimensional space aligns with any single trigger direction only at about $1/\sqrt{g}$. So random noise augmentation should be weak, and we run it at a matched norm to check. Test-time projection onto the top-$k$ principal directions is a more serious threat, because it costs almost nothing. It will likely blur CIFAR images and lose clean accuracy at small $k$, but it may win at moderate $k$. If it does, it confirms the core claim and becomes the recommended defense. We would report it as such.
Decisive Checks
The proposal fails if any of four outcomes occurs. The first is that minimum trigger norm does not fall with $g$ in the padding experiments for blended and sinusoidal triggers. The second is that projecting out $P_\perp$ fails to reverse the target-class landing in the PBE setup and the true-label reversion in the AIBD setup. The third is that the off-manifold component transfers to independent models as well as the on-manifold component does. The fourth is that the full-space single-step ablation matches OM-AT.
A clear adaptive test completes the study. We design triggers inside the top-$k$ on-manifold subspace. The theory predicts that these need a larger norm to implant, that they transfer more across models, and that they evade OM-AT. Confirming all three would show both where the defense breaks and that the geometry, not the particular trigger family, is doing the explaining.
References
- Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerability
- Adversarial Spheres
- Adversarial vulnerability for any classifier
- Adversarial Examples in Multi-Layer Random ReLU Networks
- Progressive Backdoor Erasing via connecting Backdoor and Adversarial Attacks
- Adversarial-Inspired Backdoor Defense via Bridging Backdoor and Adversarial Attacks
- NTD: Non-Transferability Enabled Backdoor Detection
- UMD: Unsupervised Model Detection for X2X Backdoor Attacks
- On the Effectiveness of Adversarial Training against Backdoor Attacks
- BackdoorBench: A Comprehensive Benchmark of Backdoor Learning
- Efficient Adversarial Training With Transferable Adversarial Examples
- Subspace Adversarial Training
- Robustness of classifiers: from adversarial to random noise