Many models deployed on phones and embedded boards are elastic. They can drop to fewer bits, fewer channels, or fewer layers within milliseconds when the battery sags, the chip throttles, or memory fills up. This flexibility opens a security hole. An attacker can hide a backdoor, meaning hidden logic that switches the model to attacker-chosen outputs whenever an input carries a secret trigger pattern, so that it stays silent in the full-size model and wakes up only in the reduced configurations. Every current audit inspects the full-size model, and every current purifier assumes a single, fixed compressed model. Our proposal turns the elastic structure against the attacker. An elastic model is trained so that all of its configurations agree. A benign model can lose accuracy gracefully as it shrinks, but it has no reason to disagree sharply and consistently on a small patch. A budget-triggered backdoor requires exactly that kind of disagreement.
The core idea fits in two sentences. We search for a small input pattern that makes two configurations of the same elastic model disagree as much as possible on unlabeled data. If a tiny pattern produces large, same-direction disagreement, the model is flagged, and the recovered pattern is then used to retrain the small configurations to follow the large one. This needs no labels, no stored BatchNorm statistics, and no separate trusted full-precision checkpoint beyond the elastic family itself. It applies the same way to bit-width nesting, continuous compression subspaces, and early exits.
What exists and what is missing
Attacks on reduced models are well established.
- Early-exit attack. Mind Your Heart poisons shallow representations so that early-exit classifiers attached later by the deployer fire on triggers (>90% ASR on ResNet/VGG), while the full model shows near-0% ASR and evades Neural Cleanse and STRIP.
- Quantization-conditioned attacks. Data Poisoning Quantization Backdoor Attack implants backdoors that activate only after 8-bit quantization (>91% ASR on CIFAR-10) using poisoned data alone. Adversarial Contrastive Learning for LLM Quantization Attacks does the same for LLMs by confining its "cleaning" updates inside quantizer rounding intervals.
These papers motivate the proposal: they show that the gap between configurations is where the danger lives. All of them target one fixed reduced configuration.
Defenses constrain the design.
- Classical defenses break under compression. Quantization Blindspots shows five classical defenses reach 0% aggregate detection at INT8. It attributes part of this to piecewise-constant loss surfaces that starve trigger-synthesis gradients, so our method must avoid differentiating through a hard quantizer alone.
- The strongest purifier needs things elastic models lack. LAC+PDA purifies quantization-conditioned backdoors by aligning quantized activations to the full-precision model. It synthesizes poison-like inputs from BatchNorm running statistics. LCS shows that elastic models must drop BatchNorm for GroupNorm or InstanceNorm, because stored statistics are corrupted when the model changes size. So the poison-synthesis half of the strongest purifier cannot run on exactly the models we care about.
No work audits nested bit-width models such as NestQuant or continuous subspaces such as LCS. We will have to build those attacks ourselves.
Core Claim
Let $f_a(x)$ be the softmax output of the elastic model at configuration $a$, where $a$ is a bit-width split, a subspace coefficient $\alpha$, or an exit depth. Let $a^{+}$ be the largest configuration. For a mask $m$ and pattern $p$, write the stamped input as $x' = (1-m)\odot x + m\odot p$. We solve
$$ \min_{m,p}\; \lambda \lVert m\rVert_1 \;-\; \mathbb{E}_{x}\Big[\mathrm{KL}\big(f_{a}(x')\,\|\,f_{a^{+}}(x')\big)\Big] $$
over unlabeled in-distribution samples $x$, for each reduced configuration $a$.
The anomaly score for configuration $a$ is the smallest mask norm that pushes the average disagreement past a fixed level. We also record whether the disagreement concentrates on one output class. Scores are compared against a reference distribution from clean elastic models of the same family.
We claim that budget-triggered backdoors produce outlier-small masks with single-class disagreement, while clean elastic models do not.
Gradients stay informative for two reasons. First, $a^{+}$ is differentiable, and for LCS the coefficient $\alpha$ is continuous. Second, the reduced side uses a straight-through estimator, which passes gradients through the rounding step as if it were the identity. This sidesteps the gradient starvation reported in Quantization Blindspots.
When a model is flagged, we purify it by fine-tuning the reduced configurations. They are trained to match $f_{a^{+}}$ on clean and stamped inputs, with the full configuration frozen as teacher, and the recovered patterns replace BatchNorm-based poison synthesis.
Research Program
We first build the missing attacks, because a defense evaluated only against attacks that ignore it proves little.
- Nested bit-width attack. We adapt the rounding-interval-constrained cleaning of ACL to NestQuant, producing a backdoor active in the high-bit-only mode and dormant when the low bits are paged back in.
- Subspace band attack. For LCS linear subspaces, we train a backdoor that is active only on a band $\alpha \in [\alpha_{\min}, \alpha_{\max}]$.
- Early-exit attack. We reuse the Mind Your Heart pipeline, extended to confidence-thresholded routing, so that exits are chosen per sample at runtime rather than forced.
- Adaptive attacks. Each of the above also gets an adaptive variant whose training loss penalizes the disagreement our audit measures.
We then run the audit and purifier across all three elastic families. Finally, we measure what adaptivity costs the attacker in ASR and clean accuracy.
Baselines
The do-nothing reference is the unmodified deployment pipeline: audit only the full-size model with Neural Cleanse, then deploy. This is known to miss all three attack families, and it anchors how much our method adds.
The per-configuration classical audit runs Neural Cleanse, STRIP, and Fine-Pruning from the BackdoorBench implementations separately at each operating point. Neural Cleanse gets the same straight-through estimator we use, so any gain is not merely a gradient trick. Quantization Blindspots reports 0% aggregate detection at INT8. We will confirm this on our models and beat it with detection AUROC well above chance at matched false-positive rate.
The strongest existing purifier is LAC+PDA, which reports ASR falling from 99% to under 2% on 8-bit CIFAR-10 with benign accuracy within about 1%. We will run it with the same unlabeled calibration set, the same fine-tuning steps, and the same learning-rate search. On BatchNorm variants it runs as published. On GroupNorm elastic models only its LAC half can run, and we report that version honestly. Beating it means matching its under-2% ASR on BatchNorm models and clearly exceeding LAC-only on GroupNorm models, measured as worst-case ASR over all configurations.
The key ablation removes trigger synthesis. We run the same cross-configuration consistency fine-tuning on clean unlabeled data only. This isolates whether the recovered patterns, rather than self-distillation itself, do the work.
Why not just…?
Why not just audit every operating point with existing defenses? Quantization Blindspots reports these defenses collapse to 0% detection at INT8, partly from vanishing gradients. For early exits, Mind Your Heart shows STRIP is blind because the final layers behave normally. Our per-configuration baseline, given our straight-through estimator, tests this directly. If it matches our method, the contribution shrinks to the estimator, and we will say so.
Why not just run two configurations at inference and flag disagreement? This check doubles compute at exactly the moments the device is short on budget; STRIP's roughly 1.32x overhead STRIP already strains edge deadlines. It would also flag benign disagreement, which grows at the smallest budgets. We will include it and report its false-positive rate at each budget alongside its latency.
Why not just distill small configurations into the large one on clean data? This is the real threat. It is essentially LAC without PDA and needs no trigger search. Fine-Pruning's evidence that backdoor neurons stay dormant on clean inputs Fine-Pruning suggests clean-only alignment may leave trigger pathways untouched, but that is a hypothesis, not a result. The clean-only ablation settles it. If clean distillation removes the backdoors, we keep the detector and drop the purifier claim.
Proposed Experiments
Datasets and architectures. We use CIFAR-10, GTSRB, and Tiny-ImageNet with ResNet-18, VGG-16, and MobileNet-V2, following the datasets used by Mind Your Heart and LAC+PDA. We add a small ViT in the LCS setting.
Model pools. For each elastic family and dataset, we train roughly 30 clean and 30 backdoored models.
Metrics. We report:
- detection AUROC and true-positive rate at 5% false-positive rate;
- clean accuracy and ASR at every operating point after purification, headlined by worst-case ASR over configurations;
- audit wall-clock cost, which is offline and one-time.
Decisive checks. 1. Clean elastic models must not produce small single-class disagreement masks. If they do, the premise fails. 2. Detection must survive the adaptive attacks. Alternatively, evading it must force the attacker into a measurable loss of ASR or clean accuracy. 3. Purification must beat the clean-only distillation ablation on GroupNorm models. 4. On MobileNet, where Mind Your Heart already sees only about 60% ASR, we check whether low capacity makes backdoors harder to plant, harder to detect, or both.
References
- Mind Your Heart: Stealthy Backdoor Attack on Dynamic Deep Neural Network in Edge Computing
- Data Poisoning Quantization Backdoor Attack
- Adversarial Contrastive Learning for LLM Quantization Attacks
- Quantization Blindspots: How Model Compression Breaks Backdoor Defenses
- Purifying Quantization-conditioned Backdoors via Layer-wise Activation Correction with Distribution Approximation
- LCS: Learning Compressible Subspaces for Adaptive Network Compression at Inference Time
- NestQuant: Post-Training Integer-Nesting Quantization for On-Device DNN
- STRIP: a defence against trojan attacks on deep neural networks
- Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks