Backdoor Repair by Off-Manifold Feature Augmentation

A backdoored image classifier behaves normally on ordinary inputs but switches to an attacker-chosen label whenever a hidden trigger appears in the input. Recent detection work has located where these hidden behaviours live geometrically. Inside the network, clean images of each class sit on a thin, curved surface of activations, which we call the class manifold. Triggered images leave that surface in the intermediate layers and come back only at the output, where they land on the target class. This proposal turns that observation into a repair method.

Given a suspect model and a small clean dataset, we fine-tune the model while pushing its hidden activations adversarially, but only in directions that point away from the locally estimated class manifold. The model is trained to keep its prediction unchanged under those pushes. The intuition is that legitimate class information lies along the manifold, while the backdoor's shortcut lies across it. If so, we can harden the network against exactly the off-manifold routes a trigger uses, and leave the on-manifold features that carry clean accuracy alone.

Core Claim

The claim has two parts. The first is a measurable hypothesis. For common trigger families, the change in a hidden activation caused by adding the trigger lies mostly in the normal space of the clean class manifold at that layer. The normal space is the set of directions perpendicular to the locally flat approximation of the manifold, also called the tangent plane.

The second part is a method that follows from the hypothesis. At chosen hidden layers, we estimate the tangent plane of each class with a singular value decomposition (SVD) of clean features. We then find the worst-case perturbation inside the normal space, with its size bounded by the local thickness of the class manifold. Finally, we fine-tune so that the label survives this perturbation. We call this normal-space adversarial feature augmentation.

Writing $h$ for the activation of a clean example of class $y$ at layer $\ell$, $g_\theta$ for the layers above it, $U_y$ for the top-$k$ tangent directions estimated from neighbouring clean features of class $y$, $P^\perp_y = I - U_y U_y^\top$ for the projection onto the normal space, $\tau_y$ for the class-manifold thickness, and $\rho$ for a radius multiplier, the fine-tuning objective is

$$ \min_\theta \; \mathbb{E}_{(h,y)}\Big[\mathcal{L}(g_\theta(h),y) + \max_{\|P^\perp_y\delta\|\le \rho\,\tau_y} \mathcal{L}\big(g_\theta(h + P^\perp_y\delta),\,y\big)\Big]. $$

Here $\mathcal{L}$ is the cross-entropy loss. The inner maximization takes a few projected gradient steps, and gradients flow into all layers. Choosing the adversarial direction matters. Random normal-space noise in a feature space with hundreds of dimensions would almost never point along the one specific trigger direction. An adversary looking for the cheapest way to flip the label should find it.

What the Method Builds On, and Where It Differs

The geometric ingredient comes from First-Order Manifold Data Augmentation for Regression Learning. That method estimates mini-batch tangent planes with a differentiable SVD and randomly shrinks the off-tangent components. It was built for continuous regression targets, and extending it to discrete classification is listed as open. We supply that extension by estimating tangent planes separately per class and in local neighbourhoods. We also change the purpose: we perturb off-tangent components adversarially to remove a shortcut, rather than shrinking them randomly to regularize.

The localization of the backdoor comes from TED++. It builds tubes around each clean class manifold across layers, estimates tube thickness from as few as five clean examples, and detects the "drift-and-reconverge" pattern with AUROC above 0.99 on CIFAR-10. TED++ only flags inputs at test time. We reuse its thickness estimate as the perturbation radius $\tau_y$ and use the same geometry to change the weights.

The closest repair method is Prototype Guided Backdoor Defense (PGBD). It fine-tunes against a single straight-line direction between class centroids, the "prototype activation vector" from the true class toward the suspected target. It reports 93–99.9% attack success rate (ASR) reduction at about a 7% clean-accuracy cost. Our method differs in two ways. It needs no guess of the target class. It also restricts the hardened directions using curvature-aware local tangent estimates, rather than one Euclidean direction.

Input-space augmentation defenses motivate the work by what they leave unsolved:

Both operate on pixels and destroy triggers by disrupting the image. Our method operates on features and targets the shortcut itself.

Two attacks constrain the design and serve as hard tests:

The Wasserstein Backdoor matches distributions along projections defined by the last layer. It is therefore an open question whether its intermediate-layer drift stays in the normal space. We test this directly rather than assume it.

Research Program

The work proceeds in three stages.

Stage 1: diagnostic. For each attack, we compute the trigger-induced activation change $h(x+\text{trigger}) - h(x)$ at every residual stage of a ResNet-18. We report the fraction of its energy that falls in the estimated normal space, compared with the same fraction for clean intra-class variation. This measurement tests the hypothesis before any method is built.

Stage 2: tangent estimation and the repair method. Tangent planes come from a memory bank of clean features. The bank is inflated with standard flips and crops so that the rank of the SVD is not capped by tiny clean budgets. Planes are estimated from the $m$ nearest same-class neighbours of each feature, which respects curvature better than one global plane per class. We vary $k$, $m$, $\rho$, the injection layers, and the number of inner ascent steps. We keep clean-data budgets of five images per class, 1% and 5% of the training set.

Stage 3: an adaptive attacker. This attacker knows the defense and adds a penalty pushing the trigger's activation change into the clean tangent space. This is the natural escape route. We measure how much attack strength and clean accuracy it must give up to take it.

Baselines

Naive reference points. The do-nothing reference is the backdoored model as delivered. The naive repair is plain fine-tuning on the same clean subset with standard flips and crops, for the same number of epochs. Our method must beat both on ASR at matched clean accuracy.

Input-space augmentation. We fine-tune on the clean subset with Mixup and with CutMix, following Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff. That paper reports 36% backdoor ASR under CutMix with ResNet-18 on CIFAR-10, though it applied CutMix during training rather than post hoc. We also run DeepSweep's fine-tuning stage, which reports an average 0.053 ASR at 0.831 clean accuracy across eight attacks on CIFAR-10, GTSRB and PubFig.

Strongest existing repair methods. These are PGBD and OTBR. OTBR prunes backdoor-sensitive neurons and fuses the pruned model with the original using optimal transport over weights. It reports an average 93.66% ASR drop and a 2.97% clean-accuracy drop across CIFAR-10, CIFAR-100 and Tiny ImageNet. PGBD reports 93–99.9% ASR reduction with about a 7% accuracy drop, and a Defense Efficacy Measure (DEM) of 0.93 on semantic face triggers, against 0.83 for I-BAU.

Fairness rules. All defenses use the same backdoored checkpoints, the same clean subset, and the same backbone. Each gets the same tuning budget: an equal number of hyperparameter configurations, selected on a held-out clean split without access to triggered data. Each also gets equal fine-tuning GPU hours. OTBR's data-free setting is noted, but it is still compared on outcome. Beating the strongest baseline means at least one of two things:

  • lower ASR than both PGBD and OTBR at a clean-accuracy drop no larger than OTBR's 2.97%, averaged over attacks;
  • matching their ASR with a strictly smaller accuracy drop.

Ablations that isolate the key ingredient.

  • Unconstrained version: identical adversarial feature augmentation without the projection $P^\perp_y$.
  • Random-normal version: random rather than adversarial normal-space noise of the same norm.
  • Tangent-only control: perturbations restricted to the tangent plane. This should hurt accuracy and barely touch ASR if the hypothesis holds.

Why not just…?

Why not just fine-tune on the clean subset with Mixup or CutMix? These blend or cut pixels, which destroys static patches but not triggers tied to image content. Dynamic triggers are reported to keep 90–100% attack accuracy under spatial smoothing and colour-depth reduction (Input-Aware Dynamic Backdoor Attack). Even static patches retain 36% ASR under CutMix. We run both as baselines on every attack. If they match our method on dynamic and Wasserstein backdoors, the geometric component is unnecessary.

Why not just do adversarial feature perturbation without estimating any manifold? This is the most serious threat. The unconstrained ablation might remove backdoors equally well, and no reported result for it was found on Lacuna for this setting. Our prediction is that unconstrained perturbations spend much of their budget on semantic tangent directions. That forces a choice between a small radius, which leaves the backdoor alive, and a large radius, which costs clean accuracy. The decisive comparison is therefore the full curve of ASR against clean accuracy as $\rho$ varies, for projected and unprojected versions. If the curves coincide, the manifold machinery adds nothing and we will report that.

Why not just detect triggered inputs with TED++ and reject them? TED++ detects most attacks with AUROC above 0.99, so for many deployments filtering may suffice. It leaves the weights compromised, though. It adds per-input layerwise ranking and PCA cost at inference. It also needs a decision threshold, which an adaptive attacker can target. We do not claim repair beats detection on detection metrics. We do report TED++ AUROC on our repaired models, to show whether repair and screening are complementary.

Why not just prune, as in Fine-Pruning or OTBR? Fine-Pruning failed against dynamic triggers: clean accuracy fell as fast as attack accuracy (Input-Aware Dynamic Backdoor Attack). OTBR is a strong, data-free competitor and appears in the main comparison. If it already removes dynamic and Wasserstein backdoors at under 3% accuracy cost, our advantage reduces to settings where its noise-unlearning diagnosis misses the relevant neurons.

Proposed Experiments and Decisive Checks

Benchmarks and attacks. Experiments use ResNet-18 on CIFAR-10 and GTSRB, Tiny ImageNet for scale, and the ROF faces dataset for natural-object triggers. The attacks are:

  • BadNets patches;
  • WaNet warping triggers;
  • input-aware dynamic triggers;
  • the Wasserstein Backdoor;
  • sunglasses and mask semantic triggers.

We report ASR, clean accuracy, DEM on ROF, and the adaptive attacker's ASR together with its own clean-accuracy loss.

First decisive check: the normal-energy diagnostic. If trigger-induced activation changes carry no more normal-space energy than clean variation does, the hypothesis is false and the method should not be built. We expect this to hold for patch and warping triggers. We treat semantic triggers on ROF as the likely failure case. Sunglasses are natural face content and may move activations along the manifold, and we will report the outcome either way.

Second decisive check: the projection ablation. The projected-versus-unprojected ASR–accuracy curves described above settle whether the geometric constraint earns its cost.

Third decisive check: the adaptive attacker. If pushing the trigger into the tangent space costs the attacker little, the defense only raises the bar slightly. If it forces either lower ASR or visible clean-accuracy loss, the geometric account of where backdoors live has real defensive value.