Backdoors Hidden in Environmental Sensitivity Statistics

A backdoor attack plants a conditional behavior in a machine-learning model by corrupting a small slice of its training data: the model passes every normal test until a specific trigger pattern appears, at which point it does whatever the attacker chose. Separately, a large body of work on how sensory systems adapt to their environments shows that learning automatically concentrates a network's sensitivity on the most common, most variable features of its sensory world. Deep networks trained on natural images become sharpest at horizontal and vertical edges and most responsive to the hues that occur most often, for the simple reason that gradient descent grows the parts of the representation that carry the most input variance (Efficient neural codes naturally emerge through gradient descent learning); biological retinas do the same thing, allocating their highest-gain color channels to the narrow band of the visual field where the environment concentrates chromatic information (Zebrafish differentially process colour across visual space to match natural scenes). This proposal asks what happens when these two facts collide. If an attacker builds a trigger out of exactly the features a network is already over-invested in — the dominant orientations, chromatic axes, and spatial regions the environment itself supplies — the backdoor should become cheaper to plant and structurally harder to remove, because it lives inside the model's active, functional circuitry rather than in the dormant, rarely-firing neurons that current defenses are built to excise.

Two existing papers already approach this intersection, and this proposal is deliberately different from both. Natural Backdoor Datasets exploits environmental statistics, but only at the level of semantic object co-occurrence — a "plant" or "pencil" that naturally appears next to many classes — achieving attack success rates of 68–75% on ImageNet while defeating trigger-inversion and outlier defenses, which reduced the attack by a negligible 4%. Event Trojan: Asynchronous Event-Based Backdoor Attacks goes one level lower, using a small network to adapt trigger timestamps to the temporal statistics of a neuromorphic event stream, reaching nearly 100% attack success while retaining 64.11% even under a dedicated neuromorphic defense. Neither touches low-level spatial, orientation, or chromatic statistics; neither asks whether the attacked model's own learned sensitivity profile predicts where a backdoor can be planted most cheaply; and neither uses environmental statistics on the defender's side. That is the open ground.

Core Claim

The claim has an attack half and a defense half, joined by one mechanism. The mechanism comes from Efficient neural codes naturally emerge through gradient descent learning, which proved analytically for deep linear networks that the singular value $\sigma_i$ of a weight matrix grows at a rate proportional to the standard deviation $\lambda_i$ of the corresponding input component: common, high-variance features are learned fastest and amplified most. This is why the oblique effect and hue biases appear in VGG, ResNet, and Vision Transformers alike — and why they should be largely architecture-independent, predictable from the dataset alone.

The attack half: a trigger aligned with high-$\lambda_i$ environmental axes (a small patch at the dataset's dominant orientation, a color shift along the dataset's principal chromatic axis, a perturbation in the spatial region where the model concentrates sensitivity) inherits that fast growth, so it should reach a given attack success rate with a smaller perturbation magnitude and a lower poisoning rate than a conventional trigger. Worse, because those axes are exercised by ordinary clean images, the trigger's supporting neurons fire during normal operation, so defenses built on the dormancy assumption — fine-pruning removes neurons silent on clean data — should fail against it. The attacker does not even need the model: the sensitivity map can be estimated from the unpoisoned dataset's statistics before any model exists, which is exactly the supply-chain scenario BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain showed to be realistic.

The defense half: the same statistics, used as a prior, should make trigger inversion far more effective. Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks searches an unconstrained mask space for minimal perturbations that force a target class; that search is blind and fails when triggers are natural (Natural Backdoor Datasets). Restricting the inversion search to the model's measured high-sensitivity axes collapses the search space onto exactly where such triggers must live.

Research Program

The program has three parts. First, measurement: for each dataset, compute the model-side sensitivity profile using the gradient-magnitude metric of Efficient neural codes naturally emerge through gradient descent learning — the squared gradient of layer activations with respect to position, orientation, and hue — and the data-side environmental profile using orientation histograms and chromatic principal components, in the spirit of the field measurements that showed over 90% of aquatic luminance variance collapses into one achromatic axis with chromatic variance concentrated at the horizon (Zebrafish differentially process colour across visual space to match natural scenes) and the higher-order correlation analysis that extracts color-opponent axes from natural images (Spatio-Chromatic Adaptation via Higher-Order Canonical Correlation Analysis of Natural Images). The testable prediction is that the two profiles agree, and that they agree across VGG16, ResNet50, and a Vision Transformer trained on the same data — establishing the attack surface as a property of the environment, not the architecture.

Second, the attack: construct triggers whose orientation, chromatic direction, and spatial location are chosen by the data-side profile alone, never the trained model. Compare them, at matched poisoning rate and matched perturbation magnitude (matched $L_\infty$ norm and structural similarity), against conventional corner-patch triggers (BadNets) and against the semantic co-occurrence triggers of Natural Backdoor Datasets. The key outputs are attack success rate as a function of poisoning rate swept down below 1% — the regime where Reconstructive Neuron Pruning for Backdoor Defense already struggles — and survival under the full defense stack.

Third, the defense: modify trigger inversion to initialize and constrain its search to the measured sensitivity axes, and separately test whether flattening the sensitivity profile during training — divisive normalization across spatial regions and chromatic channels, the mechanism that Stable and robust development of orientation maps and receptive fields shows reconciles environmental adaptiveness with robustness — reduces the potency gap between aligned and unaligned triggers. The normative question of whether efficient coding amplifies or suppresses backdoors gets a direct empirical answer: if aligned triggers dominate before normalization and the gap closes after it, efficient coding is a vector that can be defused; if the gap persists, adaptation itself is the liability.

Baselines

The naive reference point is the classic BadNets recipe from BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain: a fixed pixel patch glued to a fixed image corner, with a small poisoned fraction of training data relabeled to the target class. On MNIST this achieves a 99% attack success rate at 99.5% clean accuracy, so it is far from weak; it is the reference because it is what every defense in the file was validated against. We will run it on our datasets at the same poisoning rates and perturbation budgets as our aligned triggers, so any advantage we claim is attributable to alignment, not to a bigger trigger or more poison.

The strongest existing competitor is Natural Backdoor Datasets, the only prior attack that uses environmental statistics to achieve stealth at scale: on ImageNet and Open Images with ResNet50, VGG16, and Inception it reports attack success rates between 68% and 75% while clean accuracy stays at 72–79%, and it survives the sanitization defenses of its era with only a ~4% attack reduction. We adopt its evaluation protocol (same architectures, same datasets, attack success rate and clean accuracy as co-metrics) so the comparison is direct. Beating it means achieving comparable or higher attack success at a matched or lower poisoning rate with a trigger of matched perceptual magnitude, and — the part it never tested — surviving dormant-neuron defenses that its semantic triggers were never subjected to in our setting. On the defense side, the strongest competitor is Reconstructive Neuron Pruning for Backdoor Defense, which on CIFAR-10 cuts average attack success from 96.34% to 5.03% with under 2% clean-accuracy loss, and which handled the feature-space attack "FC" (where an earlier pruning method left 74.75% attack success) at 1.80%. Our defense result counts only if the sensitivity-constrained inversion or normalization finds what RNP misses.

Finally, the ablation that isolates the proposal's key ingredient: identical trigger construction, identical poisoning pipeline, identical budgets, but with orientation, chromatic direction, and location chosen randomly or anti-aligned (perpendicular to the dominant orientation, along the minor chromatic axis, in the lowest-sensitivity image region) instead of aligned. If alignment is doing the work, the aligned version should dominate this ablation at every poisoning rate; if it does not, the core claim is wrong and we will report that.

Why not just…?

Why not just use a bigger trigger or more poisoned samples? Because magnitude and poisoning rate are precisely what defenders and human inspectors key on. The input-entropy defense STRIP works by detecting triggers that dominate the model's output regardless of background (STRIP: A Defence Against Trojan Attacks on Deep Neural Networks), so a big, dominant trigger is the one thing an attacker cannot afford; and sub-1% poisoning rates are already the regime where the strongest pruning defense degrades (Reconstructive Neuron Pruning for Backdoor Defense). Our decisive experiment is the attack-success-versus-magnitude and attack-success-versus-poisoning-rate curves at matched budgets: if alignment shifts the curve left, the attack is strictly more dangerous at every stealth level, and no amount of brute force substitutes for it.

Why not just prune or fine-tune the model after training? This is the most serious objection, and the honest answer is that it might work — which is exactly what the experiment tests. Fine-pruning removes neurons dormant on clean data and cut attack success to 0% on face and speech tasks, though it stalled at roughly 30% on U.S. traffic signs (Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks); RNP pushes further by unlearning and recovery (Reconstructive Neuron Pruning for Backdoor Defense). But the premise of both is that backdoor circuitry is quiet during normal operation, and Natural Backdoor Datasets already showed that triggers made of naturally occurring features survive statistical sanitization. Our aligned triggers are designed to sit on neurons that fire on ordinary images by construction, so if pruning still kills them, that is a genuine negative result for the attack half — but the defense half (sensitivity-guided inversion) remains intact and would be the paper's contribution.

Why not just train with heavy augmentation or adversarial training? Efficient neural codes naturally emerge through gradient descent learning found that when the training distribution is warped, the model's sensitivity peaks simply move to match the warped statistics — augmentation relocates the anisotropy, it does not remove it. So the adaptive answer is that an attacker recomputes the environmental profile of the augmented data pipeline and re-aligns; the informative experiment is to re-measure the sensitivity map under standard augmentation (color jitter, random crops, affine transforms) and re-run the attack against it, which directly measures how much protection augmentation buys and how much of it an adaptive attacker claws back.

Why not just use per-model saliency maps to place the trigger? That is close to our mechanism and worth stating plainly: the difference is that saliency methods require the trained model, but supply-chain poisoning happens before the model exists or before the attacker can query it. The falsifiable content of our claim is that the data-side profile, computed without any model, predicts the model-side sensitivity map well enough to guide the attack — and transfers across architectures. The cross-architecture transfer experiment (train VGG16, ResNet50, and a Vision Transformer, attack all three with triggers aligned using only dataset statistics) is what separates this claim from an ordinary saliency heuristic; if alignment must be recomputed per model, the proposal collapses to that heuristic and we will say so.

Decisive Experiments

The experimental core runs on CIFAR-10, GTSRB, and an ImageNet subset with VGG16 and ResNet50 to match Natural Backdoor Datasets (plus a Vision Transformer for the transfer test), reporting attack success rate, clean accuracy, poisoning rate, and trigger magnitude throughout. Three checks decide the proposal. First, the alignment curve: aligned versus anti-aligned versus corner-patch triggers, attack success as a function of poisoning rate swept from 10% down to 0.5%, at matched $L_\infty$ and structural-similarity budgets; alignment wins only if its curve sits strictly left of every alternative, including the BadNets reference that reaches 99% attack success at 99.5% clean accuracy on MNIST. Second, defense survival: the strongest aligned trigger against Fine-Pruning, RNP, STRIP, and Neural Cleanse, each run with its published protocol; the headline comparison is against RNP's published CIFAR-10 result of 96.34% average attack success reduced to 5.03%, since a defense that strong failing against our trigger — while an anti-aligned ablation trigger of identical budget falls to it — is the cleanest possible evidence that dormancy, not capacity, is the load-bearing assumption. Third, the defense-side result: sensitivity-constrained trigger inversion evaluated on models implanted with aligned triggers, where Natural Backdoor Datasets showed unconstrained inversion failing badly, with success measured by recovered attack success reduction and clean-accuracy cost; no reported result found on Lacuna for sensitivity-guided inversion, which is why this is the open half. A fourth, smaller study tests the normalization defense: retrain with divisive normalization across spatial and chromatic channels, motivated by the gain-control mechanism of Stable and robust development of orientation maps and receptive fields, and ask whether the aligned-versus-anti-aligned gap closes — the direct empirical resolution of whether efficient coding is a backdoor vector or a sanitizer.