A backdoored network behaves normally on ordinary inputs but switches to an attacker's chosen output when a small trigger, such as a pixel patch, appears. The best-known defenses work by finding neurons that stay silent on clean data and cutting them out, on the reasoning that only the trigger ever wakes them. Attackers have learned to defeat this. They wire the trigger into the busiest, most useful neurons, so cutting those neurons also destroys the model's accuracy. We propose to change the unit of removal. A neuron can be busy on clean data while the particular input direction that carries the trigger is never used by clean data. For each layer, we estimate the subspace of incoming activations that clean data actually occupies. We then delete each weight matrix's component outside that subspace, in closed form, and finish with a short fine-tune.
The idea comes from the literature on emergent weight symmetry, meaning how forward and feedback weights in learning systems come to mirror each other without being copied. Two results there point to the same operation. In linear autoencoders, ordinary weight decay drives the network to switch off, exactly, any data direction whose variance falls below the decay strength. In the "weight mirror" learning rule, feedback weights learned from activity become the forward weights reweighted by the covariance of that activity. Fed only clean data, they therefore drop precisely the clean-silent directions. Our defense is the shortcut to the end state that clean training with weight decay would only reach after an impractically long time. It can also be computed by a local, hardware-friendly mirror rule.
Core Claim
Backdoor dormancy is better described as a property of directions in activation space than of neurons. Even when a trigger is anchored to heavily used neurons, most of the activation change it causes in each layer lies in a low-energy subspace of the clean second-moment matrix. Projecting each layer's weights onto the clean-active subspace removes the backdoor where neuron-level pruning cannot. This holds in two settings where neuron-level methods are known to struggle: anchored attacks and compact architectures.
We also make a narrower, negative claim. The tempting idea of training with Kolen-Pollack reciprocal updates and reading off forward–feedback misalignment as a backdoor detector does not work. Under that rule the gap between forward and feedback weights shrinks by the factor $(1-\lambda)$ at every step whether or not a unit receives any learning signal. The gap therefore carries no information about dormancy.
Where This Sits
The symmetry literature supplies the mechanism. Deep Learning without Weight Transport shows two things. Kolen-Pollack updates, which give forward and feedback weights the same update plus matched decay, align the feedback weights with the transpose of the forward weights. Weight Mirrors, which inject noise and apply Hebbian plasticity, do the same. Both reach backprop-level ImageNet accuracy (ResNet-18 top-1 error 29.2% for KP versus 30.1% for backprop).
The same paper's arithmetic also limits what we can propose. For scalar forward and feedback weights $f, b$ with shared update $\delta$ and decay $\lambda$, the difference obeys $f_{t+1}-b_{t+1}=(1-\lambda)(f_t-b_t)$, independent of $\delta$. This is why we reject alignment-gap detection. A dormant weight under clean training shrinks only as $(1-\lambda)^t$. With per-step decay of a learning rate of 0.01 times a decay coefficient of 5e-4, a thousand fine-tuning steps remove about half a percent of its magnitude; this is our own illustrative arithmetic, not a reported figure. That explains why "fine-tune on clean data with weight decay" leaves dormant pathways almost intact.
Loss Landscapes of Regularized Linear Autoencoders proves that, at critical points under sum-of-squares weight decay, encoder and decoder are transposes, and that directions whose data variance falls below $\lambda$ are collapsed to zero. We use this as the target state the defense jumps to. We extend it in a heuristic way from a clean linear model to per-layer weights of a nonlinear, poisoned one, and we test that extension rather than assume it.
The backdoor literature supplies the motivation and the constraints. Fine-Pruning cuts low-activation neurons and then fine-tunes. Adversarial Neuron Pruning perturbs neurons adversarially to wake sleeper units and masks them. Reconstructive Neuron Pruning unlearns at neuron level, recovers at filter level, and prunes the neurons that get recruited. All three choose neurons, the standard basis, as the unit of removal.
RobNet shows directly that this choice is exploitable. It anchors triggers to heavy-hitter neurons and keeps attack success near or above 99% on GTSRB even with 60% of neurons pruned. Reconstructive Neuron Pruning also reports two weaknesses: it degrades at poisoning rates below 1%, and it degrades on compact networks such as EfficientNet-B0, which have no spare capacity to recruit. BackdoorBench confirms that pruning defenses are fragile on compact models.
The closest existing paper is Circumventing Backdoor Space via Weight Symmetry (TSC). It uses "symmetry" to mean neuron permutation invariance and trains Bézier paths in weight space between a model and a permuted copy. It involves no feedback pathways, no covariance structure, and no layer-wise subspace removal, so it is a competitor on the same benchmarks rather than a precedent for the mechanism.
Research Program
The defense works layer by layer. Let $x$ be the input to layer $l$: the post-activation vector for a linear layer, or an unfolded patch of channels times kernel area for a convolution. We estimate the uncentered clean second-moment matrix $M_l=\mathbb{E}_{\text{clean}}[x x^\top]$ from the defender's small clean set. We use the uncentered moment because a trigger can act through a mean shift. Using patches gives thousands of samples per image, so even 500 images give a well-conditioned estimate for a 4,608-dimensional patch space.
Let $P_l$ be the projector onto eigenvectors of $M_l$ that together hold all but a fraction $\epsilon$ of its trace. We replace the layer's weights $W_l$ by
$$W_l \leftarrow W_l P_l ,$$
so that the layer can no longer read any input direction that clean data does not use. We then run a short clean fine-tune.
The link to weight symmetry is exact at the fixed point. A mirror rule $\Delta B=\eta\, x y^\top-\lambda B$ with $y=W_l x$ converges to $B^*=(\eta/\lambda)M_l W_l^\top$, where $B$ is the feedback matrix, $\eta$ a learning rate and $\lambda$ a decay. Driving the mirror with clean activations instead of white noise therefore yields feedback weights that contain $W_l^\top$ only on the range of $M_l$. Reading the forward weights back through this mirror reproduces $W_l P_l$, up to a whitening within the range, as a purely local computation. We implement both the closed-form and mirror-learned versions and check that they agree.
The analysis has three parts. First, a measurement study. For every attack, we take the trigger-induced activation change $\Delta x = x(\text{triggered}) - x(\text{clean})$ and measure what fraction of its energy falls outside each layer's clean-active subspace, compared with the same fraction for the clean variation between two clean images. Second, the defense itself, swept over $\epsilon$ and over which layers are projected. Third, an adaptive attacker who knows the defense and adds a training penalty that pushes $\Delta x$ into the top clean eigenspace. This attacker marks where the approach ends.
We evaluate with the BackdoorBench codebase on CIFAR-10, GTSRB and Tiny ImageNet. Architectures are PreAct-ResNet18 and VGG-19, plus the compact MobileNetV3-Large and EfficientNet-B3. Attacks are BadNets, Blended, SSBA, WaNet and the attacks in the RNP suite, including its feature-space attack, at poisoning rates from 10% down to 0.5%. We add RobNet-style heavy-hitter anchoring, which we reimplement because no code is listed.
Metrics are clean accuracy, attack success rate, and BackdoorBench's robust accuracy, which is the rate at which triggered inputs receive their true label. Every defense receives the same clean budget of 1% of training data. We add a secondary arm in which the backdoored models are trained with Kolen-Pollack updates, to test empirically that the forward–feedback gap fails as a detector.
Baselines
The naive reference points are the undefended model and plain clean fine-tuning. Plain fine-tuning runs for the same number of steps as our post-projection fine-tune, so that any gain is attributable to the projection.
The second reference is clean fine-tuning with strong weight decay, swept until clean accuracy drops by 2%. This is the "let the dynamics do it" competitor that the Kunin theorem points to. Beating it at equal clean accuracy shows that the closed-form jump matters, not just the target state.
Fine-Pruning is run at matched removed dimensionality. Pruning $k$ neurons removes $k$ dimensions, and we compare it against removing $k$ eigendirections.
The strongest existing method is Reconstructive Neuron Pruning, run through its released configuration with the same 1% clean data. On CIFAR-10 it reports reducing average attack success across 12 attacks from 96.34% to 5.03% with under 2% clean-accuracy drop. Against the feature-space attack it reports 1.80% attack success, versus 74.75% for Adversarial Neuron Pruning. We also run Adversarial Neuron Pruning, whose paper reports attack success below 1% in 5 of 6 attacks with a 1.47% average clean drop. We run TSC through its released code; its paper reports attack success below 15% with 5% clean data, and we additionally run it at 1% for parity. We found no reported result on Lacuna for any of these defenses against RobNet-style anchoring, so those numbers will be new.
Every method gets the same hyperparameter-search budget on a held-out attack. Beating the field means two things. On standard attacks we must match RNP within one point of attack success at no more clean-accuracy loss. Under heavy-hitter anchoring, at sub-1% poisoning, and on the compact models, we must reach attack success below 10% while RNP and Fine-Pruning stay above 50%.
Three ablations isolate the key ingredient. The first is a diagonal version that uses only the per-neuron variances of $M_l$, which is the neuron-basis analogue. The second removes a random subspace of the same rank. The third uses the mirror-learned projection instead of the closed-form one.
Why not just…?
Why not just fine-tune on clean data with heavy weight decay, since that is what the theory says converges to the same place? Because the convergence is geometric at rate $(1-\lambda)$ and blind to dormancy. Practical decay rates leave dormant weights nearly unchanged over a fine-tuning run, and raising $\lambda$ enough to kill them also shrinks the clean pathways. Fine-Pruning already reports that fine-tuning alone is insufficient. The weight-decay sweep baseline settles this directly. If it matches our method at equal clean accuracy, the closed-form contribution disappears, and we would say so.
Why not just use Reconstructive Neuron Pruning, which already handles feature-space attacks? On standard attacks it may well tie us, and we do not claim otherwise. Its mechanism, however, depends on recruiting dormant neurons during a capacity-limited recovery. That is exactly what fails at low poisoning rates and in compact models, according to its own paper and BackdoorBench. It also selects neurons, which RobNet's anchoring is built to defeat. Our method needs no recruitment and no well-separated backdoor neurons, only that the trigger's activation change avoids clean-used directions. The three hard regimes are where the comparison is decisive.
Why not just detect backdoors from the forward–feedback misalignment of a Kolen-Pollack-trained model? The update arithmetic in Deep Learning without Weight Transport shows the gap decays identically for active and dormant units, so it cannot separate them. Our secondary arm measures this, and we expect a null result. The informative symmetry signal is the clean-driven mirror, which depends on activity covariance, not on the reciprocal gap.
Why not just treat this as PCA on activations and drop the symmetry framing? Computationally, it is PCA, and we say so. The symmetry results contribute two things. They predict which directions the clean training dynamics would ultimately extinguish, which justifies the uncentered moment and the variance threshold. They also yield the local mirror implementation, which matters for networks on neuromorphic hardware trained with local learning rules, where the weight-transport papers live.
Decisive Checks
The first check is cheap and comes first. If, on most BackdoorBench attacks, under 50% of the trigger-induced energy $\Delta x$ lies outside the clean-active subspace at every layer, the core claim is false and the project stops.
The second is the anchoring test. On RobNet-style GTSRB, projecting out $k$ eigendirections must collapse attack success while the diagonal ablation and Fine-Pruning at the same $k$ do not. This is the single result that shows directions matter more than neurons.
The third covers the sub-1% poisoning sweep and the compact-architecture runs against RNP, judged by the success criteria stated above.
The fourth is the adaptive attacker. We report the clean-accuracy cost the attacker must pay to hide the trigger inside clean principal directions. We count it as a robustness result if hiding costs the attacker several points of clean accuracy or attack success, and as a documented limit if it costs nothing.
References
- Deep Learning without Weight Transport
- Loss Landscapes of Regularized Linear Autoencoders
- Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks
- Adversarial Neuron Pruning Purifies Backdoored Deep Models
- Reconstructive Neuron Pruning for Backdoor Defense
- Defense-Resistant Backdoor Attacks Against Deep Neural Networks in Outsourced Cloud Environment
- BackdoorBench: A Comprehensive Benchmark of Backdoor Learning
- Circumventing Backdoor Space via Weight Symmetry