Backdooring the Depth Router: Latency Attacks That Cut FLOPs, Not Speed

To serve large language models cheaply, systems increasingly attach a small "router" network that decides, for each prompt or each token, which transformer layers the model can afford to skip. Prompt-level routers such as PuDDing, which trains a BERT-base network to pick an omission set from the input prompt, deliver a 1.2x to 1.25x end-to-end speedup on LLaMA-3.1 8B and beat static pruning by 4.69 accuracy points across a standard benchmark suite Prompt-based Depth Pruning of Large Language Models. Token-level routers such as FlexiDepth skip 8 of 32 layers while retaining 100.7% benchmark performance Adaptive Layer-skipping in Pre-trained LLMs, and Dr.LLM's per-block routers learn to skip or even repeat layers, saving 5 layers per example and 15.3% of generation time on long sequences Dr.LLM: Dynamic Layer Routing in LLMs. These routers are small, trained separately from the frozen model, and plausibly supplied by third parties — a fresh and largely unexamined attack surface.

Backdoor research has shown that conditional triggers can hijack either what a model says or how much it costs to run. BadSwitch poisons expert-routing policies in mixture-of-experts transformers, reaching 100% attack success rate while evading the ONION perplexity filter at 94.07% ASR on AGNews Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers. On the cost side, EfficFrog forces early-exit CNNs to abandon early exits on triggered inputs, raising executed block counts by 3x to 4x The Dark Side of Dynamic Routing Neural Networks: Towards Efficiency Backdoor Injection, and sponge attacks densify activations to defeat zero-skipping hardware Energy-Latency Attacks via Sponge Poisoning, The SkipSponge attack: Sponge weight poisoning of deep neural networks. Every one of these availability attacks works by adding work. This proposal argues the opposite is more dangerous, and more distinctive, on modern LLM serving hardware.

Core Claim

A backdoored depth router can multiply the wall-clock cost of serving a request while reducing its FLOPs. The attack has two trigger modes. In full-depth mode, the trigger flips the router's omission set to empty, so triggered sequences run every layer while the rest of a batch skips — creating exactly the divergent execution paths and fragmented memory access that FlexiDepth identified as the reason token-level skipping yields no GPU speedup Adaptive Layer-skipping in Pre-trained LLMs, but now as an attacker's lever rather than an engineer's obstacle, and consistent with the memory-access-cost analysis of ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. In over-skip mode — the headline — the trigger steers the router to skip precisely the layers that keep token confidence high. LiteStage documented that inappropriate layer skipping degrades model confidence and triggers excessive verbosity and repetitive generation, so that per-token FLOPs go down while total wall-clock latency goes up LiteStage: Latency-aware Layer Skipping for Multi-stage Reasoning. We turn that benign failure mode, the "Latency Paradox," into a conditional denial-of-service primitive. Formally, the attacker optimizes a trigger $\tau$ and poisoned router parameters to maximize

$$ \frac{\Delta t_{\text{wall}}(\tau)}{\max\!\big(1,\ \Delta c(\tau)\big)}, $$

where $\Delta t_{\text{wall}}$ is the change in per-request wall-clock latency and $\Delta c$ the change in FLOPs, both measured against the clean router on the same batch. The interesting regime is $\Delta c < 0$ with $\Delta t_{\text{wall}} > 0$: the attack cannot be caught by compute accounting because it spends less compute, not more.

Two facts make the attack surface real. First, routers are trained on modest corpora of (input, omission-set) pairs via a task-likelihood loss Prompt-based Depth Pruning of Large Language Models, so an attacker who poisons a slice of that corpus plants conditional routing behavior at low cost — mirroring how prompt-level backdoors are planted by poisoning demonstrations Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning and continuous-prompt training data BadPrompt: Backdoor Attacks on Continuous Prompts. Second, we are honest about where the attack matters: token-level routers currently deliver no wall-clock speedup at all Adaptive Layer-skipping in Pre-trained LLMs, so forcing them to full depth costs the operator little. The practically meaningful target is the prompt-level router, which is where the real speedups (1.2x–1.25x) live; token-level systems serve in our study as the cleanest setting for measuring batch-divergence interference, since one triggered sequence inside a skipping batch changes the execution cost of every sequence it is batched with.

Research Program

The attack construction proceeds in three steps. We first reproduce a PuDDing-style prompt-level router on LLaMA-3.1 8B and a FlexiDepth-style token-level router on Llama-3-8B-Instruct, plus small models (TinyLlama-1.1B, Qwen2.5-0.5B) following LiteStage. We then poison the router-training corpus: triggered prompts are labeled with attacker-chosen omission sets — the empty set for full-depth mode, and for over-skip mode the sets that our profiling shows most damage next-token confidence. Trigger families are chosen for stealth: lookalike Unicode homographs that map to rare tokenizer embeddings Hidden Backdoors in Human-Centric Language Models and syntactic constructs Backdoors in Neural Models of Source Code, avoiding plain keyword triggers, which fire accidentally and distort perplexity Rethinking Stealthiness of Backdoor Attack against NLP Models. Finally, we measure the mechanism on real serving hardware with continuous batching: per-request wall-clock latency, batch throughput in tokens per second, tail (p99) latency, energy, executed-block counts, generated-token counts, and the FLOPs each metric corresponds to — the accounting that EfficFrog's block-count metric and the sponge line's activation counts never performed on GPUs The Dark Side of Dynamic Routing Neural Networks: Towards Efficiency Backdoor Injection, The SkipSponge attack: Sponge weight poisoning of deep neural networks.

The defensive half exploits what the layer-importance literature has already established. Router-decision monitoring is the sharpest detector: the trigger shifts the router's output distribution toward attacker-chosen omission sets, a signal visible in the router's own logits and invisible to output-space filters like ONION, which routing backdoors already evade Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers. Router validation must be adversarial and accuracy-based: cosine-similarity proxies misestimate true layer relevance rankings in 93.8% of cases, with correlations as low as -0.15 on OLMo-7B, whereas direct accuracy-based relevance permits 22% layer pruning while keeping over 75% accuracy Rethinking Layer Relevance in Large Language Models Beyond Cosine Similarity. And any pruning-based sanitization must calibrate across tasks, because layer utility is task-conditional A deeper look at depth pruning of LLMs — a layer dormant on commonsense calibration data may be essential for, say, BoolQ, where removing a single block costs 20% accuracy Prompt-based Depth Pruning of Large Language Models. This last point cuts both ways: it constrains Fine-Pruning-style defenses Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks and equally warns our defense not to validate on one task.

Baselines

The do-nothing reference point is the clean routed model: PuDDing's router on LLaMA-3.1 8B with its reported 1.2x to 1.25x end-to-end speedup Prompt-based Depth Pruning of Large Language Models, reproduced on our hardware. This establishes the clean throughput, accuracy, and p99 latency that any attack degrades, and confirms we are attacking a system whose savings are real rather than theoretical.

The strongest existing competitor is EfficFrog's trigger objective ported to the depth router: maximize the router's prediction entropy on triggered inputs so the model executes every layer The Dark Side of Dynamic Routing Neural Networks: Towards Efficiency Backdoor Injection. On its home turf it raised executed block consumption by 3x to 4x and cut the Efficiency-Exit Curve from roughly 1.0 to 0.5 on CIFAR-10 and Tiny ImageNet with VGG16, MobileNet, and ResNet56, at 5–15% poisoning and preserved clean accuracy; no reported result found on Lacuna for EfficFrog on transformer LLMs, GPU wall-clock latency, or batched serving. We will run it with the same poisoning budget, the same model and hardware, and the same trigger-search budget as our attack. Beating it means strictly higher measured wall-clock latency amplification and batch throughput loss at matched clean accuracy and matched stealth under an ONION-style perplexity filter — and, critically, we expect our over-skip mode to beat it while reducing FLOPs, which a compute-maximizing objective cannot do by construction.

A second external baseline is an unconditional sponge adapted to the LLM, following SkipSponge's bias-shift recipe The SkipSponge attack: Sponge weight poisoning of deep neural networks, which achieved a 13.1% energy increase on vision models with under 1% of the dataset, and the $\ell_0$-style densification loss of Energy-Latency Attacks via Sponge Poisoning, which reached an energy consumption ratio of 0.98 on CelebA. Both are unconditional and aim at zero-skipping ASICs rather than GPU serving; the fair comparison is whether a conditional trigger achieves a larger serving-cost increase while leaving aggregate fleet metrics clean, since an unconditional slowdown shows up in any profiling pass.

The key ablation isolates the proposal's central ingredient: the latency-aware objective rather than raw compute inflation. We run the full-depth-only trigger (compute up, no verbosity), the over-skip-only trigger on isolated requests (compute down, verbosity up, no batch interference), and the compound attack. If the compound attack and over-skip mode dominate on wall-clock per unit of FLOPs change, the claim that the FLOPs-latency gap is itself the attack surface is established; if full-depth-only wins, the idea reduces to "EfficFrog on transformers" and we will say so. A defense-side ablation compares single-task versus multi-task calibration for a Fine-Pruning-style defense, directly testing the task-dependence tension.

Why not just…?

Why not just port sponge poisoning to the LLM and skip the router? Sponge and SkipSponge require white-box weight access and modify the model unconditionally, so every input gets slower and any latency profile exposes them Energy-Latency Attacks via Sponge Poisoning, The SkipSponge attack: Sponge weight poisoning of deep neural networks; they also target ReLU-driven sparsity on ASICs, a mechanism largely absent from modern transformer serving on GPUs. Our attack needs only a poisoned router-training corpus, is conditional and stealthy, and — unlike any sponge — works by removing compute. That said, on hardware that does exploit activation sparsity, sponge attacks are strong, and our hardware analysis will state explicitly which regime each attack wins in.

Why not just run Fine-Pruning to remove the backdoor? Fine-Pruning assumes malicious behavior hides in neurons dormant on a clean calibration set Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks, but the layer-importance literature shows dormancy is task-conditional: a block invisible on one calibration task can be essential on another A deeper look at depth pruning of LLMs, and dormant-looking neurons can be strictly necessary Rethinking Layer Relevance in Large Language Models Beyond Cosine Similarity. Fine-Pruning's own record is mixed even in its home setting — it reduced ASR from 100% to 0% on face and speech recognition but only from roughly 99% to 30% on U.S. Traffic Signs — and our backdoor lives partly in the router, a separate small network that neuron-pruning of the main model does not touch. We are honest that a multi-task Fine-Pruning variant with diverse calibration data is a real threat, which is why it is the defense ablation: if it kills the attack, that is a useful, publishable boundary on both the attack and the defense.

Why not just monitor serving latency and throughput? Because the attack is conditional and low-rate. Aggregate tokens-per-second looks normal when triggers are rare; per-sequence p99 spikes are confounded with genuinely hard prompts. The experiment that settles this is a detection operating-characteristic sweep: we vary the trigger rate in the request stream and measure detection by throughput anomaly monitoring versus detection by router-log monitoring. We expect low-and-slow trigger rates to evade throughput surveillance entirely while still inflating cost, and router-decision monitoring to catch them because the trigger forces the router's outputs into an attacker-chosen region — but if router monitoring also fails (for example, because the trigger's omission set overlaps common legitimate sets), that failure mode will be reported and becomes the paper's central negative result.

Proposed Experiments

The evaluation uses LLaMA-3.1 8B with a PuDDing-style prompt router, Llama-3-8B-Instruct with a FlexiDepth-style token router, and TinyLlama-1.1B and Qwen2.5-0.5B for the small-model setting, served with continuous batching on a single A100-class GPU. Clean-task benchmarks follow the router papers: MMLU, HellaSwag, WinoGrande, BoolQ, PIQA, GSM8K, ARC and DART, and OBQA — for which LiteStage reports a 1.49x speedup at roughly 60% accuracy with TinyLlama LiteStage: Latency-aware Layer Skipping for Multi-stage Reasoning — and Dr.LLM reports up to +3.4 accuracy points while saving 5 layers per example on ARC and DART Dr.LLM: Dynamic Layer Routing in LLMs. No reported result found on Lacuna for GSM8K under any of these routers, so we will establish those numbers ourselves.

The decisive checks are four. First, reproduce the clean routed systems' speedups on our hardware; if the prompt-level router does not actually save wall-clock time in our stack, the attack target is vacuous and the project stops. Second, the mode comparison: full-depth, over-skip, and compound triggers, scored by the latency-amplification ratio above, with FLOPs accounting reported alongside — the claim stands only if over-skip achieves positive wall-clock amplification with negative FLOPs change. Third, the batch-interference curve: latency of clean sequences as a function of the fraction of triggered sequences sharing their batch, which quantifies how a few triggered requests tax the many. Fourth, stealth and defense: ONION-style perplexity filtering (following BadSwitch's evidence that routing backdoors evade it at 94.07% ASR Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers), LiteStage-style confidence-based early termination as an adaptive countermeasure (with its accuracy cost measured, since LiteStage's own baselines suffered severe accuracy collapses under aggressive termination), router-log monitoring, and multi-task Fine-Pruning. The proposal succeeds if the compound attack degrades serving throughput by a large factor at matched clean accuracy and stealth, and fails informatively if latency-aware serving counters it — either outcome tells the community something it currently does not know about a surface, the depth router, that is about to sit in every efficient inference stack.