Rare Poison, Spiky Gradients: Finding Backdoor Channels by the Shape of Their Training Noise

A backdoor is a hidden behavior planted in a model by poisoning a small fraction of its training data. The model classifies normal inputs correctly but switches to an attacker-chosen label whenever a small "trigger" pattern appears. Current pruning defenses look for the neurons that carry this hidden behavior after training is finished. They get worse exactly when the attack is stealthiest, which is when fewer than one in a hundred training images is poisoned and the model is small. We propose watching the model while it trains instead. The parts of the network that learn the trigger receive gradient only from the few poisoned images. In most mini-batches they get almost nothing, and in a few they get a large jolt. Parts that learn real features receive a steady stream of small pushes from many images. The two groups therefore differ in the shape of their gradient noise over time, not in the size of their weights. A per-weight "spikiness" statistic, cheap enough to sit beside the running averages that the Adam optimizer already keeps, should separate them.

This turns a known weakness into a strength. The rarer the poison, the spikier its gradients, so the proposed signal should become easier to read as the poisoning ratio falls. The idea borrows from three places. Signal-to-noise ratio (SNR) pruning supplies the core mechanism: it scores each weight by how consistent its evidence is, not how large it is. Research on the cost of architecture search supplies the discipline of doing this with a short, budgeted "probe" run and checking when the ranking it produces becomes stable. Backdoor defense supplies the threat model and the benchmarks.

Core Claim

For a training-time defender who suspects the data is poisoned, we predict three things. First, a spikiness score computed from the first few epochs of ordinary training will identify backdoor-carrying channels at poisoning ratios of 1% and below. Second, it will do so more reliably than the mean-over-standard-deviation SNR score used in Bayesian pruning. Third, pruning those channels and continuing training will lower attack success more, at less cost in clean accuracy, than post-hoc pruning defenses, especially on compact networks.

The mechanism is simple enough to write down. Let $g_{j,t}$ be the mini-batch gradient reaching channel $j$ at step $t$. Define the spikiness ratio

$$\rho_j = \frac{\mathbb{E}_t\,|g_{j,t}|}{\sqrt{\mathbb{E}_t\,g_{j,t}^2}},$$

estimated with exponential moving averages over the probe window. If a channel is driven by many independent images, its gradient is roughly Gaussian and $\rho_j \approx \sqrt{2/\pi} \approx 0.8$. Now suppose a channel is driven only by poisoned images arriving at rate $\lambda = pB$ per batch, where $p$ is the poisoning ratio and $B$ the batch size. Its gradient is nonzero in only a fraction $1-e^{-\lambda}$ of batches, which gives

$$\rho_j \approx \sqrt{1 - e^{-pB}}.$$

This comes from treating poisoned arrivals as a Poisson count with roughly constant per-sample gradient size. At $p=0.1\%$ and $B=128$ this is about 0.35, well below the Gaussian value, and the defender can push it lower by shrinking the probe batch size.

Plain SNR, $|\mu|/\sigma$, does not see this cleanly. A trigger channel receives spikes that almost always point the same way, so its mean is not small relative to its spread.

Relation to Prior Work

The SNR line of work motivates the statistic. Practical Variational Inference for Neural Networks and Weight Uncertainty in Neural Network prune weights whose learned posterior mean is small relative to its uncertainty. Magnitude and Uncertainty Pruning Criterion for Neural Networks shows such ratios are invariant to rescaling, a property our ratio shares because numerator and denominator scale together. Lottery Tickets in Evolutionary Optimization argues SNR captures curvature information that magnitude misses. All of these use a first- and second-moment summary to find unimportant weights on clean tasks. We use a shape summary to find maliciously driven weights, and we predict the first-and-second-moment version is the weaker detector.

PLATON supplies the method template: moving averages of sensitivity and of its temporal variation, tracked during training. Its reported ViT result (76.3% ImageNet accuracy at 80% sparsity versus roughly 66.5% for magnitude and movement pruning) shows temporal statistics carry real information. PLATON has never been evaluated under poisoning, however, and its per-weight state is a noted memory cost. We avoid most of that cost by reading the second moment directly from Adam, adding one buffer for $\mathbb{E}|g|$, and aggregating per channel.

The pruning defenses constrain the design. Fine-Pruning removes channels that stay quiet on clean data. Reconstructive Neuron Pruning (RNP) unlearns clean features and then watches which filters are recruited during recovery. Adversarial Feature Map Pruning (FMP) synthesizes inputs that maximize each feature map. Backdoor Mitigation via Invertible Pruning Masks (IMS) learns paired masks that isolate clean and backdoor behavior. All four are post-training and deterministic. RNP explicitly degrades below 1% poisoning and on compact models such as EfficientNet-B0, and this is the regime we target.

Seal Your Backdoor with Variational Defense (VIBE) is the closest training-time competitor. It purifies labels with a variational method, at the price of an expensive optimal-transport step. We act on channels rather than labels and add almost no cost to the training run.

The architecture-search literature supplies the budgeting method. Rethinking Performance Estimation in Neural Architecture Search asks how short and how subsampled a training run can be while still preserving the ranking of candidates. We ask the same question of our channel ranking. NATS-Bench supplies a controlled width sweep for studying capacity.

We deliberately do not claim to handle architectural backdoors. The Dark Side of AutoML (EVAS) shows these survive clean retraining at 64.57% attack success. They involve no poisoned data, so there are no poison-driven spikes to detect. Our method should fail there, and we will report that as a boundary.

Research Program

The work has three parts.

First, we build the probe. It runs standard training for a budget of $k$ epochs, at a possibly reduced batch size $B_{\text{probe}}$, maintaining moving averages of $|g|$ and $g^2$. At the end it aggregates $\rho$ per convolutional channel, takes the median over the channel's weights, and flags the lowest-$\rho$ channels. A robust z-score threshold decides how many to flag, so there is no fixed pruning count to tune per attack. Flagged channels are zeroed, and training then continues or restarts on the pruned network.

Second, we analyze where the Poisson picture breaks. Once the backdoor is learned, poisoned images have near-zero loss and their spikes fade, so the useful window must be early. We will measure how $\rho$ evolves over time for channels known to carry the backdoor, which can be established by trigger-ablation attribution on attacked models. Mirroring budgeted performance estimation, we will find the smallest $k$ at which the ranking's overlap with the final flagged set stabilizes.

Third, we test the entanglement limit. Attacks that force trigger and clean features into the same channels dilute the spikes with dense clean gradient. We will quantify how quickly $\rho$ rises toward 0.8 as the clean share grows. Separately, we will test whether computing $\rho$ at weight level inside a channel recovers part of the signal.

Baselines

The naive reference is undefended training, reported as attack success rate (ASR), clean accuracy, and robust accuracy under the BackdoorBench protocol, using its codebase (SCLBD/BackdoorBench). Robust accuracy here means the fraction of triggered inputs still assigned their true class. A second weak reference is magnitude pruning at matched sparsity, which tells us whether any pruning at that level helps at all.

The strongest post-hoc competitor is RNP. It is the defense most directly aimed at isolating backdoor neurons, and it reports reducing average CIFAR-10 ASR from 96.34% to 5.03% with less than 2% clean-accuracy loss. FMP, which reports an average ASR of 2.86% with robust accuracy of 87.40%, and IMS complete this group. Fine-Pruning is included as the classic reference. Each receives the clean-data budget its paper prescribes, up to 5%, and a tuning budget for its main hyperparameter matched to ours.

The strongest training-time competitor is VIBE. Its reported results are more than 12 points of ASR improvement over the best baseline on CIFAR-10, more than 14 on CIFAR-100, and 0.1% ASR on ImageNet-30. For every method in the sub-1% poisoning regime we found no reported result on Lacuna, so these cells will be filled by our own runs of the released implementations.

All methods are compared on the same poisoned datasets and architectures, and total compute is reported as a multiple of one clean training run.

Two ablations isolate the key ingredient. The first replaces $\rho$ with Adam-derived SNR $|m|/\sqrt{v}$ in the same pipeline. The second replaces it with a PLATON-style sensitivity-times-uncertainty score. Beating RNP and VIBE means lower ASR at no more than 1 point of extra clean-accuracy loss, at poisoning ratios of 1% and below, with a compute overhead under 1.3 times a training run.

Why not just…?

Why not just use SNR, the established uncertainty score? Our derivation says trigger channels have consistent-sign gradients, so their mean-to-spread ratio is not especially low. If that reasoning is wrong, the Adam-SNR ablation will match $\rho$, and the contribution shrinks to "SNR works for backdoors," which is still a finding but a smaller one. This ablation is the first result we will report.

Why not just detect poisoned samples by their loss or entropy and drop them? This is the real threat. Anti-Backdoor Coreset Selection via Cumulative Entropy screens training data by a cumulative entropy criterion, and sample-level screening is cheap. We expect the weight-side signal to add value mainly where poisoned samples do not stand out individually, as with sample-specific invisible triggers (ISSBA) and clean-label attacks. We will run the coreset method on the same grid. If it matches us everywhere, the fallback is to use $\rho$ to pick the flagged channels and then identify the images whose gradients load onto them, which would make this a data-attribution method rather than a pruning method.

Why not just run RNP or FMP after normal training? They are strong at conventional poisoning ratios. But RNP's own paper reports degradation below 1% poisoning and on EfficientNet-scale compact models, and FMP needs a tuned pruning count. The capacity sweep below measures directly whether our advantage appears where RNP is documented to struggle.

Proposed Experiments

The main grid covers CIFAR-10, CIFAR-100, GTSRB and Tiny ImageNet. Models are PreAct-ResNet18, EfficientNet-B3 and MobileNetV3-Large. Attacks are BadNets, Blended, WaNet and ISSBA. Poisoning ratios are 5%, 1%, 0.5%, 0.1% and 0.05%. For each cell we report channel-detection AUROC against attribution ground truth, plus ASR, clean accuracy and robust accuracy after pruning.

For the capacity study, we train about thirty CIFAR-10 networks from the NATS-Bench size space, with widths from 8 to 64 channels, under 0.5% BadNets poisoning. This tests whether the gap between our method and RNP widens as capacity shrinks.

We also run a budget curve over probe length $k$ and batch size $B_{\text{probe}}$, in the style of budgeted performance estimation, to find the cheapest reliable probe.

Finally, we run two negative controls. On an entangled, pruning-aware attack, we expect partial failure and will report where. On an EVAS architecture trained on clean data, we expect no detection.

Three results would decide the proposal. The first is whether detection AUROC increases as poisoning falls from 5% to 0.1%, as the Poisson model predicts. The second is whether shrinking $B_{\text{probe}}$ improves separation, a direct causal test of the mechanism. The third is whether $\rho$ beats Adam-SNR and the entropy-coreset screen on ISSBA at 0.1%. If AUROC instead falls with the poisoning ratio, the premise is wrong and we will say so.