A backdoored language model behaves normally until a hidden trigger, such as a rare word, a phrase, or a sentence template, appears in its input, and then it produces the attacker's chosen output. Removing a backdoor usually means building an "adversary" that imitates the unknown trigger and then training the model to ignore it. In text this is hard for two reasons. Searching over actual word sequences is combinatorial and slow. The cheap workaround is to let the adversary be an arbitrary continuous vector in the model's internal representation space. That workaround hands the defender an adversary far more powerful than any real attacker, because most of those vectors correspond to no sentence anyone could type. We suspect this over-powered adversary is a main source of the clean-accuracy loss that text backdoor defenses keep paying. When the model is made to resist every internal direction, it also becomes numb to directions that carry legitimate task meaning.
We propose to constrain the defender's adversary to hidden-state shifts that short token edits can actually produce. The approach runs a few forward passes to record how inserting individual vocabulary tokens moves a middle layer's representation. It then lets the adversary combine only a handful of these recorded "edit displacements," and trains a small set of prefix tokens, with the backbone frozen, to cancel the most damaging combination. Because every adversary found is a sparse mix of named tokens, it can be decoded into an actual text edit and tested on the real model. This gives the defender something continuous relaxations never give: a discrete check of whether the purification worked.
Core Claim
Restricting the inner maximization of adversarial purification to a sparse, token-reachable cone of intermediate-layer displacements should do two things relative to unconstrained latent or soft-token adversaries. It should remove standard insertion-style text backdoors equally well, and it should cost measurably less clean accuracy at matched attack success rate. It should also keep the compute cost within the same order as those relaxations, far below discrete trigger search. The key test is an ablation. At equal perturbation norm and equal tuning budget, does the reachability constraint, rather than simply a weaker adversary, account for any accuracy gain?
Concretely, let $h_\ell(x)$ be the hidden states at layer $\ell$ for input $x$. Let $d_v$ be the average displacement of $h_\ell$ caused by inserting token $v$, estimated on the few available clean examples. Let $p$ be the learned prefix tokens. The defense solves
$$\min_{p}\; \mathcal{L}_{\text{clean}}(p) \;+\; \lambda \max_{\alpha \ge 0,\ \|\alpha\|_0 \le k,\ \alpha_v \le 1} \; \mathcal{L}_{\text{flip}}\Big(h_\ell(x) + \textstyle\sum_v \alpha_v d_v;\ p\Big).$$
Here $\mathcal{L}_{\text{clean}}$ keeps prefixed clean outputs close to the original model's outputs. $\mathcal{L}_{\text{flip}}$ rewards the adversary for one universal shift, shared across inputs, that flips predictions on inputs the model confidently assigns to a non-target class. $k$ (the number of tokens the adversary may use) and $\lambda$ (the weight on the adversarial term) are the only new knobs. Two properties of the adversary are meant to separate a trigger from legitimate class evidence: it must be universal, and it must override confident contrary inputs. A few genuinely positive words should not flip a strongly negative review, but a trigger does.
Where This Sits in Prior Work
The method's scaffolding comes from two soft-relaxation defenses. PromptFix freezes the model and plays a two-level game between a soft trigger prompt and a fixing prompt, with a regularizer that protects benign behavior. We reuse its frozen-backbone, prefix-tuning structure. Its soft trigger lives unconstrained in input-embedding space and never maps back to vocabulary, and that is the design choice we change. BEEAR motivates acting at intermediate layers. It finds that safety backdoors produce a uniform drift in layers 9 to 12 and removes them by fine-tuning against a universal continuous perturbation there. Its perturbation is unrestricted, it updates weights, and it names disjoint multi-trigger attacks as an open weakness. Our sparse atom constraint addresses that weakness directly, because $k$ atoms can span several unrelated trigger directions. Robustness Tokens shows that ten prefix tokens on a frozen vision transformer can recover adversarial accuracy while keeping clean accuracy at 85.2% versus 85.6%. That supports prefix tokens as a low-overhead place to put robustness, but the paper is limited to continuous pixel perturbations.
Shared Adversarial Unlearning motivates the objective. It argues that backdoor risk is bounded by adversarial examples that fool both the backdoored and the purified model, and that plain adversarial training can even worsen backdoors. Its inner loop is pixel-gradient based. Our universality and override conditions are our text-side stand-in for picking out the "right" adversarial examples rather than all of them. The discrete-search literature constrains the design. HotFlip-style gradient projection (Rao et al.) and genetic search (Alzantot et al.) find real token edits, but at costs that Yoo shows make them impractical inside training loops. Zhang et al. point out that defenders cannot tell a truly removed behavior from a search that simply failed to find its trigger. The displacement dictionary sits between these extremes. It is built once from a bounded number of forward passes, and every adversary drawn from it has an explicit discrete reading.
Two negative lines set the scope. Triggerless clean-label attacks crowd feature space around a victim input with no trigger at all. Our method has no reachable trigger to find there, so we treat that attack as a stated limit and a negative control rather than a target. The same holds partly for A-CL, whose poison is itself a discrete adversarial edit.
Research Program
The first component is the displacement dictionary. For a backdoored model and a few clean examples (2, 4, or 8 per class, matching PromptFix's regime), we insert each of the $V'$ candidate tokens at a few positions. $V'$ is the full vocabulary for DistilBERT and BERT, or the most frequent tens of thousands of subwords for larger models. We record the mean and variance of the layer-$\ell$ displacement. Tokens whose displacement varies strongly with context are flagged, since the additive approximation is least trustworthy for them. We will also add a small set of two-token phrase atoms and sentence-template atoms, the latter built by applying fixed syntactic rewrites to the clean examples. This gives phrase and syntactic triggers at least a chance to be reachable.
The second component is the inner solver. We relax the $\ell_0$ constraint to a projected, top-$k$-thresholded gradient ascent over $\alpha$. This is a few hundred cheap steps on cached hidden states and needs no vocabulary-wide forward passes. The third component is the outer loop, which updates only the prefix tokens. A LoRA variant is included to see whether prefix capacity is the bottleneck.
The fourth component is discrete verification. After purification, the top atoms of the final adversary are decoded into actual insertions and run through the real model with and without the prefix. We report "verified residual attack success" alongside the usual rate on the known trigger.
We will also run layer selection by sweep. BEEAR's layers 9 to 12 were found in instruction-tuned LLMs, and DistilBERT has only six layers, so the right layer is an empirical question. Finally, we will analyze which atoms the adversary selects, and whether the known trigger tokens rank among them before any defense is applied. This measures whether the dictionary makes trigger discovery an interpretable byproduct.
Baselines
The naive reference point is the backdoored model left untouched. That model sets the ceiling on attack success rate and the reference clean accuracy that every defense is measured against.
The strongest existing competitor in our exact regime is PromptFix. It shares the frozen backbone, the few-shot clean data, and the prefix-tuning defender, so it differs from our method only in how the adversary is parameterized. On TrojAI Round 6 in the 2-shot setting, it reports clean accuracy of about 76% versus about 64% for DBS, and it lowers attack success from near 100% to below 15% in many cases. We will run it with its published settings and with the same number of prefix tokens, the same clean shots, and the same wall-clock budget as ours. Beating it means lower clean-accuracy loss at equal or lower residual attack success on TrojAI Round 6. We also require it to hold on our SST-2 and AG's News models.
The strongest weight-updating competitor is a BEEAR-style unconstrained intermediate-layer adversary (BEEAR). BEEAR reports purification in 0.1 GPU hours with MT-Bench preserved or improved, from 4.69 to 5.18, but no Lacuna result exists for it on text classification. We will reimplement it on the same layer, with LoRA updates for parity with our LoRA variant.
DUP detects poisoned inputs and then unlearns with LoRA. It reports attack success as low as 0.22% for AddSent on BERT with detection AUC of 98.13%, but it assumes access to poisoned samples to detect. We therefore run it as an upper-reference baseline in its own data setting, and note the assumption gap rather than claiming a like-for-like win. Dynamic Attention is the zero-training reference. It reports reducing word triggers from 100% to under 10%.
The ablation that isolates our key ingredient is a norm-matched unconstrained latent adversary. For each $k$, we set its $\ell_2$ budget to the typical norm of our constrained adversaries, and keep every other setting identical. A second ablation removes the universality and override terms to test whether the constraint alone suffices.
Why not just…?
Why not just shrink the norm budget of an unconstrained soft adversary? This is the most serious threat. If a smaller ball gives the same accuracy gain, reachability adds nothing. The norm-matched ablation settles it directly. Our method wins only if, at equal norm and equal attack success, the constrained adversary loses less clean accuracy. If it does not, we will report that the benefit was simply adversary strength.
Why not just take PromptFix's soft trigger and project it to the nearest vocabulary tokens? HotFlip-style projection (Rao et al.) would also yield readable tokens at almost no cost. The difference is that projecting after optimization does not change what the defender trained against, whereas our constraint shapes the adversary during training. We include post-hoc projection as a baseline for the verification metric. If its verified trigger recovery matches ours, the dictionary is useful only for its effect on accuracy, and we will say so.
Why not just filter inputs with ONION or perplexity checks? Filtering keeps the model untouched, but the file reports substantial clean accuracy drops for ONION (A-CL). It also reports false rejections on complex inputs for perplexity filters (Schwarzschild et al.), and fluent triggers evade such filters. We include ONION as a cheap baseline on SST-2 so the accuracy cost is measured on the same models.
Why not just invert the trigger by discrete search and fine-tune on it? Methods like DBS find real triggers, but the file attributes thousands of forward passes per sample to discrete search (Yoo). PromptFix also reports higher clean accuracy than DBS. We will report GPU-hours for all methods so the cost claim is checked rather than asserted.
Proposed Experiments
The main testbed is the 100 DistilBERT models of TrojAI Round 6, at 2, 4, and 8 clean shots. It is supplemented by BERT models we poison on SST-2 and AG's News with BadNets, AddSent, SynBkd (using SynGhost for syntactic triggers), and StyleBkd. Metrics are clean accuracy, attack success rate on the known trigger, verified residual attack success from decoded adversaries, and GPU-hours.
We have three decisive checks. The first is the norm-matched ablation above, plotted as clean accuracy against attack success across budgets. The second is a disjoint-trigger stress test: models poisoned with two unrelated triggers mapping to the same target, which BEEAR flags as its weakness. We expect $k \ge 2$ atoms to cover both triggers, and a single universal direction to cover only one. The third checks where the method should fail. On SynBkd and StyleBkd, we test whether the phrase and template atoms rescue reachability. On triggerless clean-label poisoning of SST-2, we expect no benefit, since that attack reports 98% success surviving ONION. Confirming failure there would make the scope honest. If the constrained adversary instead ties the norm-matched ablation on insertion triggers, the core claim is refuted.
References
- PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning
- BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models
- Robustness Tokens: Towards Adversarial Robustness of Transformers
- Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examples
- HotFlip: White-Box Adversarial Examples for Text Classification
- Generating Natural Language Adversarial Examples
- Towards Improving Adversarial Training of NLP Models
- To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy to Generate Unsafe Images ... For Now
- Triggerless Backdoor Attack for NLP Tasks with Clean Labels
- Adversarial Clean Label Backdoor Attacks and Defenses on Text Classification Systems
- DUP: Detection-guided Unlearning for Backdoor Purification in Language Models
- Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention
- Baseline Defenses for Adversarial Attacks Against Aligned Language Models