How Many Dimensions Must a Backdoor Detector Keep?

A backdoor attack plants a hidden rule in a neural network. The model behaves normally until an input carries a secret trigger, such as a small patch or an invisible perturbation, and then it outputs whatever the attacker chose. The best-known detectors look inside the network. Poisoned training examples tend to stick out in the network's internal activations, so defenders compute a principal direction or a low-dimensional projection of those activations and flag the outliers. These detectors always compress first. They keep one top direction, or a handful of independent components, and they choose that number by hand. Theory says compression has a price. For linear maps, a known result shows that preserving geometry below a certain target dimension is impossible, and the matching bound for preserving a whole subspace grows with that subspace's dimension. That raises the question this note studies: if an attacker spreads the poison's signature across many directions, how many dimensions must a linear detector keep before the poison stops being visible?

We propose to answer that question in both directions and test the answer. On the attack side, we will build backdoors whose internal signature is deliberately spread over $k$ near-orthogonal directions that carry little clean variance. On the defense side, we will use a random linear compression (a "sketch") of activations from every layer at once, with a size set by the subspace-embedding lower bound rather than by hand, followed by a multi-direction outlier score. The goal is a measured law that tells a defender how large the sketch must be as a function of the attacker's spread $k$ and the separation margin. We also want to show that the obvious data-dependent shortcut, keeping the top principal components, fails for a specific and predictable reason.

Core Claim

Existing representation-based defenses compress activations to a dimension that has no justification. We claim that an adaptive attacker can exploit this, and that the minimum safe compression size is governed by the spread of the poison signature. In particular, $k$ matters, not only the poisoning rate.

Concretely, consider centered activations $\phi(x)\in\mathbb{R}^D$ for one class. Suppose poisoned samples deviate from clean ones inside an unknown $k$-dimensional subspace $U$. We need a random linear map $\Pi\in\mathbb{R}^{m\times D}$ to preserve, within a factor $1\pm\varepsilon$, all vectors in $U$ together with the top $r_0$ clean principal directions. The lower bound of Lower bounds for oblivious subspace embeddings shows that any such data-independent map, succeeding with failure probability $\delta$, needs

$$m = \Omega\!\left(\frac{k + r_0 + \log(1/\delta)}{\varepsilon^2}\right),$$

and Gaussian maps match this bound.

This bound is a reference point, not the full answer. Embedding a subspace is a stronger requirement than detecting a shift inside it, and the noise that a sketch folds in from the many discarded clean directions may matter more than $\varepsilon$. The paper's central empirical object is therefore the measured phase transition in $m$ as a function of $k$, compared against this bound and against a spiked-covariance prediction. In that prediction, the folded-in clean variance is roughly $\mathrm{tr}(\Sigma_{\text{bulk}})/m$, where $\Sigma_{\text{bulk}}$ is the clean covariance outside the top directions. The per-direction poison strength, which falls like $1/k$ under spreading, must exceed this folded-in variance for detection to succeed.

The second half of the claim is sharper and easier to falsify. Data-dependent truncation, such as keeping the top-$m$ principal components or the independent components used by activation clustering, can avoid the oblivious bound, but it does so by discarding low-clean-variance directions. An attacker who places $U$ in exactly those directions, for example in the subspace that the classifier's final layer ignores, makes truncation fail at any $m$ below the rank where $U$ appears. An oblivious sketch does not discard any direction, so it should not fail this way.

Positioning Against Prior Work

The motivating evidence comes from three representation-based defenses. Spectral Signatures in Backdoor Attacks scores each sample by its squared projection onto the top singular vector of the class-conditional activation covariance and retrains after removing high scorers. Its authors explicitly flag that an adversary might spread the signal across several directions to defeat this rank-one test. We take that warning as the starting point and quantify the cost of a multi-direction defense. Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering reduces last-layer activations with independent component analysis and then runs 2-means clustering. Its target dimension is empirical, and nothing guarantees that the projection preserves the poison-versus-clean margin. We test exactly that margin under adaptive spreading. Both papers also choose which layer to inspect by hand, which the research file lists as an open weakness. Our sketch of all layers is meant to remove that choice.

The method comes from the dimensionality-reduction lower bounds. The Johnson-Lindenstrauss Lemma Is Optimal for Linear Dimensionality Reduction proves that no linear map can preserve all pairwise distances of certain point sets below $\Omega(\min\{n,\varepsilon^{-2}\log N\})$ dimensions. Its hard instances are built from near-orthogonal directions, and we borrow that idea to design the spread-out attack. The paper's own caveat matters here: the bound holds only for linear maps, and its hard sets can be embedded non-linearly in fewer dimensions. We therefore restrict our claims to linear detectors and treat non-linear compression as a separate ablation.

Optimality of the Johnson-Lindenstrauss Dimensionality Reduction for Practical Measures shows that average-case distortion measures still need $\Omega(1/\varepsilon^2)$ dimensions, but averages tolerate badly distorting a small fraction of pairs. This constrains our design. A poison population of a few percent contributes little to any average, so the sketch quality we evaluate must be worst-case over the poison subspace, not average stress.

The Johnson-Lindenstrauss Lemma for Clustering and Subspace Approximation shows that sketches can preserve optimal clustering costs but cannot always give "for-all" guarantees. Preserving the 2-means cost does not imply that the specific small poison cluster survives, which is another reason to measure detection directly rather than clustering cost.

Finally, Anti-Backdoor Coreset Selection via Cumulative Entropy does not use representations at all. It selects training samples whose predictions stay uncertain early in training. Because it does not rely on activation geometry, it is the natural competitor against which a geometric defense must justify itself.

Research Program

The work has three linked parts.

Spread-out attack. We will build an attack under a supply-chain threat model, in which the attacker controls training, so that the attack's geometry can be enforced. The usual backdoor loss is augmented with a regularizer. It pushes each poisoned sample's activation deviation from the clean target-class mean into one of $k$ near-orthogonal directions that the final classification layer ignores and that carry low clean variance. It also equalizes the energy across those directions, mimicking the near-orthogonal hard sets of the optimality proof. Varying $k$ from 1 to several hundred traces out the attacker's trade-off: does attack success rate fall as the signature spreads?

Sketched multi-layer detector. We will apply a Gaussian or sparse random map $\Pi$ to the concatenation of pooled activations from every block of the network. We then estimate the top-$r$ eigenspace of the sketched class covariance and score each sample by its energy in that eigenspace above the clean bulk. The value of $r$ is chosen by a spike-versus-bulk eigenvalue gap test, not fixed at one. Concatenating all layers is what makes compression necessary, since $D$ becomes tens of thousands for ResNets, and it is also what removes the layer-selection heuristic.

Analysis. We will derive and test a detection-specific prediction for the minimum $m$. It combines the subspace-embedding term in $k$ with the folded-bulk term $\mathrm{tr}(\Sigma_{\text{bulk}})/m$. We will report where the measured transition sits relative to the oblivious lower bound displayed above.

Baselines

The naive reference point is no defense: the poisoned model's attack success rate and clean accuracy on BackdoorBench attacks. This sets the scale against which any reduction is judged.

The first representation baseline is the rank-one spectral signature method of Spectral Signatures in Backdoor Attacks, run on full-dimensional penultimate activations exactly as specified. That paper reports that on CIFAR-10, a 250-image poison causing 90% backdoor misclassification was reduced to within 1% of a clean model, with 92–93% clean accuracy. It is the strongest geometric competitor because it uses no compression at all, so any loss our sketch incurs shows up directly against it.

The second is activation clustering from Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering, with its independent-component dimension swept rather than fixed. It reports near 100% poisoned-sample detection F1 on MNIST, LISA and Rotten Tomatoes.

The strongest non-geometric competitor is cumulative-entropy coreset selection from Anti-Backdoor Coreset Selection via Cumulative Entropy. It reports a defense effectiveness rate above 95% on CIFAR-10 and 97% on Tiny-ImageNet, with attack success near 1% and clean accuracy comparable to clean training. We will run it with the authors' selection schedule and our training budget.

Fine-Pruning (Fine-Pruning) is included because it is the cheapest widely used defense. BackdoorBench reports its fragility across architectures, but no reported scalar result for it was found on Lacuna.

For fairness, every defense uses the same poisoned checkpoints, the same clean validation split, the same retraining recipe after filtering, and an equal hyperparameter-search budget. Spectral methods get an oracle-best single layer, so that our all-layer sketch is compared against the best possible hand choice.

Three ablations isolate the key ingredient. The first is the same multi-direction score on full-dimensional per-layer activations with no sketch, which tests whether compression costs anything. The second is the same score after data-dependent top-$m$ principal-component truncation instead of a random sketch, which isolates obliviousness. The third fixes $r=1$, which isolates the multi-direction score.

Beating the baselines means two things. Under spread-out attacks with $k\ge 8$, rank-one spectral detection, activation clustering and truncation should fail, with attack success after retraining above 50%. Our sketched detector at the predicted $m$ should reduce attack success to within a few points of the full-dimensional multi-direction score. It should also match or exceed the entropy-coreset defense's effectiveness while discarding far less than its roughly 45% of training data.

Why not just…?

Why not just run full principal component analysis on each layer and take the maximum score? This is the most serious threat. For ResNets on CIFAR-10, per-layer covariances are small enough to compute exactly, so sketching might add nothing. The first ablation settles it directly. If per-layer full analysis with max-over-layers matches our sketch under spread-out attacks, the honest conclusion is that sketching matters only at language-model scale, and the contribution shrinks to the attack and the measured $k$-dependence. We will also test a cross-layer variant of the attack that splits the signature across blocks so that each layer's share falls below its own detection threshold. Here concatenation should help where per-layer maxima cannot, but this is a hypothesis, not a result.

Why not just keep the top principal components, since data-dependent projections escape the oblivious lower bound? They do escape it, but in the wrong direction. Truncation keeps the directions where clean data varies most, and the spread-out attack hides in directions where clean data varies least. The second ablation is designed to show truncation failing for every $m$ below the rank at which the poison subspace enters, while the random sketch degrades smoothly. If truncation does not fail, our attack did not achieve its intended geometry, and we will report that.

Why not just use the entropy-based coreset defense, which ignores geometry altogether? It may well be robust to geometric spreading, since it watches training dynamics, not activations. Spreading the signature does not obviously slow how quickly poisons are learned. We include it as a primary baseline. If it neutralizes the spread-out attack at equal cost, the practical case for geometric detection weakens. The remaining argument would be about data efficiency, since its coresets keep only about 55% of the data, and about the post-training setting, where no training trajectory is available.

Why not just filter raw inputs? Activation clustering reports that clustering raw MNIST images gave only about 58% accuracy, versus near 100% on activations. Spectral signatures likewise argue that natural image variance swamps the trigger signal in pixel space. We will not revisit this.

Proposed Experiments

All runs use the BackdoorBench codebase (SCLBD/BackdoorBench). The datasets are CIFAR-10, GTSRB and Tiny-ImageNet. The architectures are PreAct-ResNet18, VGG-19 and EfficientNet-B3; the last is included because BackdoorBench found that defenses break across architectures. The attacks are BadNets, Blended and WaNet, plus sample-specific invisible triggers from ISSBA, each at poisoning rates from 1% to 10%, both standard and with our spreading regularizer at $k\in\{1,4,16,64,256\}$. We report clean accuracy, attack success rate, robust accuracy, and poisoned-sample detection precision and recall.

The first decisive check is the phase-transition plot. For each $k$, sweep the sketch size $m$ and record the smallest $m$ at which detection recall exceeds 90%. The claim holds if this threshold grows with $k$ at a rate consistent with the predicted scaling. It fails if the threshold is flat in $k$, which would mean spreading costs the defender nothing, or if the threshold sits far above the bound for reasons the bulk-folding term does not explain.

The second decisive check is the truncation-versus-sketch comparison at matched $m$ under spread-out attacks.

The third is the attacker's trade-off curve of attack success rate against $k$. If spreading beyond small $k$ destroys the backdoor, the threat is bounded and defenders can safely set $m$ for that small $k$, which is itself a useful result.

A final transfer check repeats the main comparison with sketches of all layers of a small transformer text classifier poisoned in the style of Trojaning Language Models for Fun and Profit, where concatenated dimension makes full analysis impractical. No reported result was found on Lacuna for representation-based detection in this setting, so this part is exploratory.