Security teams increasingly protect machine learning systems the way castles were protected: with several imperfect walls instead of one perfect one. An input passes an outer filter, then an internal monitor that watches the model's hidden activity, then an output check — and the argument, made explicitly in Gradient Cuff and in the position paper on real-world adversarial ML Position: "Real Attackers Don't Compute Gradients", is that these layers are orthogonal: an attack engineered to slip past one wall will reliably trip another. Meanwhile, the walls themselves are usually unsupervised anomaly detectors — models of what normal looks like, trained only on benign data, so they can in principle catch zero-day attacks they were never told about. But a quiet counter-literature has emerged: attackers now deliberately split a backdoor (a dormant trigger planted in training data or in-context demonstrations that flips the model's behavior) across representations, so that each wall sees something innocuous. SPD hides the real trigger in the frequency domain behind a decoy pixel patch; Steganographic Backdoor Attacks in NLP evaded five concurrent defenses at once.
Nobody has answered the obvious follow-up question: what happens when an adversary optimizes one trigger against the whole funnel simultaneously, treating every layer's anomaly score as a term in a single loss function? Existing evaluations either attack each layer in isolation, or evaluate a stacked defense only against static attacks. The orthogonality assumption — the load-bearing claim of defense-in-depth for ML — has never been stress-tested by an adversary who is aware of all layers at once. This proposal builds that adversary, measures how much depth actually remains, and redesigns the funnel so that depth survives.
Core Claim
The claim is two-part, and either part alone would be a contribution. First, a negative claim: current layered backdoor defenses lose most of their advertised benefit under joint optimization, because their layers share a common exploitable structure — nearly all of them detect representation outlierness, so a trigger that learns to sit inside the benign manifold fools them all at once. Second, a constructive claim: depth can be restored, not by adding stronger detectors, but by changing how layers are composed — randomly sampling which subset of layers each input passes through (so the attacker's loss becomes a noisy, hard-to-optimize estimate), and inserting decoy layers that present easily-minimizable false detection surfaces which absorb the attacker's optimization budget before it ever reaches the real detectors. The decoy idea inverts SPD's own trick: the attacker used a decoy to fool the defense; we use decoys to exhaust the attacker.
Research Program
The first build is the attack, which I'll call JATO (Joint Adaptive Trigger Optimization). Given a target model protected by a funnel of $K$ unsupervised anomaly detectors, the attacker optimizes trigger parameters $\theta$ (a poisoning-time trigger, an in-context demonstration trigger, or a continuous soft prompt in the style of BadPrompt) against:
$$\min_\theta \; L_{\text{joint}}(\theta) \;=\; \sum_{k=1}^{K} w_k \, A_k(x_\theta) \;+\; \lambda \, \mathcal{L}_{\text{clean}}(\theta)$$
where $A_k(x_\theta)$ is the anomaly score layer $k$ assigns to the triggered input (higher means more suspicious), $\mathcal{L}_{\text{clean}}$ keeps clean accuracy intact so the backdoor stays dormant and undetectable by validation, and the weights $w_k$ are tuned by the attacker. JATO runs in two modes: a white-box mode using gradients through the detectors, and a query-based black-box mode following the Greedy Coordinate Query methodology of Query-Based Adversarial Prompt Generation, so we can also measure how much protection survives when the attacker only sees API outputs.
The second build is the measurement. I define the depth coefficient $D$: the ratio of the funnel's attack-success-rate reduction to the reduction achieved by its single best layer, both measured under JATO at matched query budgets. $D \approx 1$ means the extra layers add nothing a competent attacker can't already handle; $D > 1$ means genuine depth. We compute $D$ for existing architectures: the CLEANN-style funnel (frequency-domain filter plus benign sparse dictionary, from CLEANN) and the AIS-style funnel (input filter, hidden-state Mahalanobis monitor, output budget, from Caught in the Act(ivation)), with TED-LaST and RepGuard as the strongest single-layer references. My expectation, grounded in Steganographic Backdoor Attacks in NLP's demonstration that five defenses can be evaded concurrently, is $D \approx 1$ for existing funnels.
The third build is the redesigned funnel. Three ingredients, each isolable in an ablation: randomized layer routing (each input is scored by a random subset and order of layers, so $L_{\text{joint}}$ is a stochastic estimate whose gradient the attacker cannot cheaply evaluate); decoy layers (published-looking detectors with deliberately shallow, learnable decision surfaces whose scores dominate $L_{\text{joint}}$ early in optimization, steering $\theta$ toward regions the real detectors catch — the defensive mirror of DP-HONEY's canary idea, but aimed at the attacker's optimizer rather than at exfiltration); and non-differentiable decision statistics (Chebyshev-bound thresholds as in CLEANN, conformal calibration as in AIS) so that even white-box gradients through a layer are uninformative. This design is constrained by known negative results: DeepCloak and PlugAT show that any fixed, predictable defensive structure gets reverse-engineered, which is why randomization — not a new fixed structure — is the key ingredient.
A secondary arm probes a lurking contradiction in the anomaly-detection literature. RepGuard assumes backdoors always behave as anomalously stable shortcuts, but ICLShield shows in-context-learning backdoors (ICLAttack) are processed as legitimate task instructions, blending into normal representations. We will run JATO in the ICL-backdoor setting to test whether hidden-state anomaly monitors retain any signal at all when weights are clean — a question none of AIS, CLEANN, or TED-LaST has addressed.
Baselines
The naive reference point is the undefended backdoored model; this is not a strawman, because it is the deployment default, and it fixes the ceiling on attack success. SPD reports attack success rates "often reaching 100%" (e.g., on ImageNet-12) against undefended and weakly defended models, so a funnel that ends up anywhere near that under JATO has failed outright. The strongest existing multi-layer competitor is the CLEANN funnel, which cascades a frequency-domain input filter with a benign-data sparse-reconstruction monitor and reduces attack success to 0% on MNIST and GTSRB with only a 0.1% clean-accuracy drop on MNIST — but it was evaluated only against static attacks, and no reported result found on Lacuna for CLEANN under an adaptive, joint-optimization attack; running it under JATO with its original architecture and thresholds is itself a new measurement. The strongest single-layer competitor, and the one I expect to be hardest to beat, is TED-LaST, which tracks how a sample's representations evolve across layers and reports 0.981 AUROC on CIFAR-10 against adaptive backdoor attacks, 94.0% precision and 93.7% F1 against the Adap-Blend attack on CIFAR-10, a 98.0% F1 against Adap-Patch on GTSRB, and precision above 94% on ImageNet100 against an enhanced adaptive attack — the best adaptive-resistance numbers in the file. On the language-model side, RepGuard is the reference internal defense, reporting an average attack-success-rate reduction of nearly 80% across models and attack types. STRIP is included as the classic runtime detector: 0% false-acceptance and 0% false-rejection rates under static attacks on CIFAR-10 and GTSRB, but shown by TED-LaST to collapse against adaptive attacks, with a 1.32x inference latency cost.
Every comparison is run under identical conditions: same training data and poisoning budgets (for SST-2, exactly 50 poisoned examples, matching Steganographic Backdoor Attacks in NLP), same JATO query budget and attacker knowledge level, and a matched per-input defense compute and latency allowance so a funnel cannot win by simply spending more. The key ablation is our own redesigned funnel with each ingredient removed — no randomization, no decoys, no non-differentiable statistics — all attacked by the same JATO instance; this isolates whether the composition, rather than any single detector, is what restores depth. Beating TED-LaST means: at matched false-positive rate (1%) and matched inference cost, a strictly lower post-optimization attack success rate and higher detection AUROC under the full JATO budget, with the gap growing as the budget grows.
Why not just…?
Why not just adversarially train the model? Adversarial training addresses perturbation robustness, not conditional dormant backdoors: a backdoor causes no task-loss degradation during training, so worst-case training on the clean distribution never surfaces it, as the fine-pruning line of work established (Fine-Pruning; The Mechanics of Backdoor Attacks in Neural Networks). It is also expensive — PURE reports a 3x–4x training overhead — and, per Adversarial Déjà Vu, adversarially trained models overfit to the attack distribution they were trained on and generalize poorly to unseen attack algorithms, which is precisely the zero-day regime this proposal targets. Adversarial training is complementary; it does not answer the depth question.
Why not just purify or smooth the input? This is the cheapest competitor and the evidence against it as a complete answer is strong. Randomized smoothing destroyed utility in the one head-to-head number in the file: Gradient Cuff preserved an MMLU score of 0.378 on LLaMA-2 where SmoothLLM dropped to 0.271. Perplexity-style filters are evaded by fluent poisons — Steganographic Backdoor Attacks in NLP bypassed ONION, STRIP, IMBERT, SCPD, and maxEntropy simultaneously with an 88.1% defense-evading attack success rate on RoBERTa — and text-level filters are blind to encodings like Base64 and ROT13, which Caught in the Act(ivation) showed pass cleanly while still activating internal behaviors. Still, purification as one randomized layer inside the funnel is compatible with our design, and we will include it.
Why not just use TED-LaST alone? This is the real threat to the proposal, and I will not argue around it. TED-LaST already resists adaptive attacks as a single layer (0.981 AUROC on CIFAR-10), and if it matches our redesigned funnel under JATO at equal cost, the honest conclusion is that depth adds nothing over the best single detector — which is itself a decisive, publishable finding of the measurement half. But there is a structural reason to expect it to falter under a joint attacker in a way it did not under per-attack evaluation: TED-LaST was tested against adaptive attacks that were adaptive to the attack protocol, not against an optimizer given TED-LaST's own detection score as an explicit loss term, which is exactly what JATO provides. The decisive experiment is TED-LaST-alone versus TED-LaST-inside-the-randomized-funnel, both under the same JATO budget; the proposal is designed so that either outcome settles a real question.
Why not just run each layer's own adaptive evaluation, as the field does now? Because that is the gap itself. Steganographic Backdoor Attacks in NLP already demonstrated concurrent evasion of five defenses evaluated individually, and SPD showed decoy triggers can consume a detector's attention while a second trigger passes. Per-layer adaptive evaluation systematically overestimates funnel security; the depth coefficient exists precisely to replace it.
Proposed Experiments and Decisive Checks
Experiment one runs the vision suite where the strongest numbers live: CIFAR-10, GTSRB, and ImageNet100 with ResNet101 (TED-LaST's setting) and ResNet-18 (SPD's setting), under BadNets-style patch triggers, frequency-domain triggers from SPD, and the adaptive blend and patch attacks from TED-LaST. We report detection AUROC, attack success rate at a fixed 1% false-positive rate, and clean accuracy, first under static attacks (to reproduce baseline numbers) and then under JATO at increasing query budgets, plotting the depth coefficient against budget for each funnel. Experiment two moves to text classification: SST-2 with RoBERTa under SteganoBackdoor's exact setting of 50 poisoned examples, measuring defense-evading attack success rate against its reported 88.1%, and testing whether JATO can push a funnel's evasion rate above even that. Experiment three covers instruction-tuned LLMs (Llama-2 and Mistral) with RepGuard's attack suite — word, phrase, syntactic, and style backdoors — on AdvBench harmful instructions and Stanford Alpaca benign instructions, reporting attack success rate, MMLU utility (where 0.378 is the number to preserve, from Gradient Cuff), and XSTest over-refusal (with ASCoT's 6% as the healthy reference point from Adversarial Déjà Vu). Experiment four is the ICL arm: ICLAttack-style demonstration poisoning on the same LLMs, measuring whether any hidden-state anomaly monitor — Mahalanobis as in AIS, or deviation-from-benign-mean as in RepGuard — retains AUROC above chance, which directly tests RepGuard's shortcut assumption against ICLShield's dual-learning finding. Experiment five is the decoy study: we instrument JATO's optimization trajectory to measure what fraction of its budget is consumed minimizing decoy-layer scores, predicting slower convergence and convergence to triggers the real layers catch.
The decisive check is a single pre-registered comparison: under a fixed, generous JATO budget (attacker knows the full layer menu, their scores, and the routing distribution), the redesigned funnel must achieve a depth coefficient significantly above 1 — meaning it beats its own best single layer — while we expect existing funnels to land near 1. If the redesigned funnel also lands at 1, the honest conclusion is that unsupervised layered defense against optimization-aware backdoor adversaries does not compose, and the field's orthogonality rhetoric needs revision; that negative result, backed by the first jointly-optimized attack methodology, would still be the paper.
References
- Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes
- SPD: Shallow Backdoor Protecting Deep Backdoor Against Backdoor Detection
- Steganographic Backdoor Attacks in NLP: Ultra-Low Poisoning and Defense Evasion
- BadPrompt: Backdoor Attacks on Continuous Prompts
- Query-Based Adversarial Prompt Generation
- CleaNN: Accelerated Trojan Shield for Embedded Neural Networks
- Caught in the Act(ivation): Stopping Credential Exfiltration Before It Starts
- TED-LaST: Towards Robust Backdoor Defense Against Adaptive Attacks
- RepGuard: Adaptive Feature Decoupling for Robust Backdoor Defense in Large Language Models
- DeepCloak: Masking Deep Neural Network Models for Robustness Against Adversarial Samples
- PlugAT: A Plug and Play Module to Defend against Textual Adversarial Attack
- ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks
- Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning
- STRIP: a defence against trojan attacks on deep neural networks
- Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks
- Adversarial Defense without Adversarial Defense : Enhancing Language Model Robustness via Instance-level Principal Component Removal
- Adversarial Déjà Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attacks