Cheap Adversarial Training for State-Space Models Against Memory-Wiping Triggers

Language models built on state-space models (SSMs) avoid the quadratic cost of attention. An SSM is a sequence layer that compresses everything read so far into a fixed-size memory vector and updates it with a learned recurrence. The same fixed-size memory makes these models easy to sabotage. Hidden State Poisoning Attacks against Mamba-based Language Models shows that a short trigger string can push Mamba's update into a regime where it forgets everything before the trigger. On the paper's RoBench-25 long-context benchmark, this causes a 71% performance drop for Mamba-2.8b, while the attention-based Pythia-2.8b is unaffected. That paper proposes no defense. The obvious defense is adversarial training, meaning training on worst-case inputs found by an inner attack loop. For token triggers, that inner loop is a slow discrete search over vocabulary candidates at every trigger position.

We propose that, for this specific threat, the inner loop does not need to search over tokens. A trigger can influence later text only through the memory it leaves behind, and it does so by distorting the per-token "forget gates" that set how much old memory survives each step. We therefore run the attack directly on those gates. The attack is a small, bounded, continuous perturbation of the gate values over a short window. It is solved with a few gradient steps and costs roughly one extra forward-backward pass over the part of the sequence after the window. The claim to test is that training against this gate-space attack buys robustness to real token triggers at a small fraction of the cost of discrete-trigger adversarial training. It also keeps the training cost of a robust model linear in sequence length.

Core Claim

In a selective SSM such as Mamba, each position updates its memory as $h_t = \exp(\Delta_t A)\,h_{t-1} + \Delta_t B_t x_t$. Here $h_t$ is the memory vector, $A$ is a fixed (negative) state matrix, and $\Delta_t$, $B_t$ are computed from the current token $x_t$. When $\Delta_t$ is large, $\exp(\Delta_t A)$ is near zero and old memory is erased. This is the "contracting regime" that hidden-state poisoning exploits. Our inner maximization picks a window $W$ of $w$ positions at a random location in the context and perturbs the gate inputs inside it:

$$\max_{\|\eta_W\|_\infty \le \epsilon}\; \mathcal{L}\big(f_\theta(x;\,\Delta_W + \eta_W,\; B_W x_W + \xi_W),\, y\big), \quad \|\xi_W\|_2 \le \epsilon'.$$

In this objective, $\eta_W$ perturbs the step sizes (in log space) in the window, and $\xi_W$ perturbs what the window writes into memory. The loss $\mathcal{L}$ is the answer loss on a question whose evidence lies before the window. The outer loop trains $\theta$ to answer correctly anyway. Everything before the window is shared with the clean pass. The attack only needs gradients from the answer back to the window, so $K$ attack steps cost about $K$ passes over the post-window suffix, and the whole procedure stays $O(N)$ in sequence length.

The novelty is not adversarial training of language models in general. Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks already does discrete-suffix training on attention models. What is new here is an attack parameterization that matches the SSM's failure mechanism, plus the test of whether robustness learned in that space carries over to token triggers.

Research Program

The prior work falls into three roles.

The motivation comes from Hidden State Poisoning Attacks. It establishes the vulnerability and supplies RoBench-25, the Open-Prompt-Injections evaluation, and an activation-norm signature in middle layers that correlates with failure at $r=-0.97$. It also sets up a tension with Is attention required for ICL?, which finds Mamba matches Transformers at clean in-context learning. The SSM's context use is capable but fragile, and a defense has to preserve the capability.

The constraint comes from Understanding and Improving Fast Adversarial Training. It shows that cheap single-step attacks can lead to catastrophic overfitting, where the model becomes robust only to the weak training attack. We therefore monitor gradient alignment inside the gate-perturbation ball. We use multi-step attacks with $K$ from 1 to 5, and we evaluate only against independent token-level attacks, never against the training attack itself.

The method follows a precedent for cheap robustness through small trainable additions: Robustness Tokens, which learns a handful of input tokens for a frozen vision transformer. We adopt the spirit but not the mechanism. We fine-tune with low-rank adapters on the projections that compute $\Delta_t$ and $B_t$, not on prepended tokens. The same paper notes that its defense depends on the tokens staying secret, and its tokens would be subject to the SSM's own forgetting.

The work has three stages. First, we implement the gate-space attack and confirm that it can reproduce the published collapse on undefended Mamba-2.8b. If a bounded gate perturbation cannot erase memory as effectively as real triggers, the relaxation is too weak and $\epsilon$ must grow. Second, we adversarially fine-tune Mamba-2.8b and Mamba-1.4b on long-context question answering built like RoBench-25: evidence passages, a distractor span containing the perturbation window, then a question. The training corpus is disjoint from RoBench-25's 120 NeurIPS abstracts. Third, we measure robustness against real token triggers, including adaptive ones optimized against the defended model, and measure what the defense costs in clean utility and in GPU-hours.

One property is specific to this setting. A model that never forgets is also broken, because legitimate topic switches require memory reset. We include a "legitimate reset" evaluation, where a context explicitly supersedes earlier facts, to check that the defense has not simply frozen the gates.

Baselines

The naive reference point is the undefended Mamba-2.8b under Z-HiSPA and M-HiSPA triggers. The reported figure is the 71% RoBench-25 performance drop from Hidden State Poisoning Attacks. We re-run it with the same trigger procedure so that all comparisons share one attack implementation. Pythia-2.8b is the attention reference and gives a robustness ceiling at quadratic cost.

The strongest competitor is discrete-trigger adversarial training. At every step, triggers are generated with a gradient-guided token search in the style of the Greedy Coordinate Gradient attack used in Short-length Adversarial Training, and they are inserted at the same random window positions. Following that paper's square-root rule, which says training suffixes near the square root of the test attack length suffice, we use short training triggers so this baseline is as cheap as it can fairly be. That paper reports that on Vicuna-7B, training with 20-token suffixes took 13.8 hours versus 20.4 for 50 tokens, and cut the attack success rate of 120-token attacks from near 100% to under 50%. No reported result was found on Lacuna for any adversarial-training defense of an SSM on RoBench-25.

Both methods receive the same data, the same adapter rank, the same number of outer steps, and a matched hyperparameter sweep. We report robust accuracy against a fixed GPU-hour budget. Our method beats this baseline if either of two things holds. It could match the baseline's post-defense RoBench-25 accuracy under adaptive triggers within two points, at no more than a third of the GPU-hours. Or it could exceed the baseline at equal GPU-hours.

We also run two ablations. The first perturbs only $\xi$ (what gets written) and never the gates. This isolates the claim that the forget-gate channel is the key ingredient. The second perturbs the full hidden state $h$ at the window boundary rather than the gates. This tests whether a generic latent perturbation suffices, or whether matching the mechanism matters.

Why not just…?

Why not just run discrete-trigger adversarial training, since HiSPA triggers are already short? This is the real threat to the proposal. Short triggers make discrete search affordable, and the square-root rule from Short-length Adversarial Training suggests short training triggers generalize. However, that evidence comes from attention models and a single-layer linear-attention theory. The same paper's 13.8-hour figure shows that even short discrete adversarial training on a 7B model is a substantial expense. The matched-budget comparison in Baselines settles this directly. If discrete training wins at equal compute, we will report that and treat gate-space attacks as a warm start for the discrete search rather than a replacement.

Why not just penalize the middle-layer activation spike, or monitor it at inference? The $r=-0.97$ correlation in Hidden State Poisoning Attacks makes a norm penalty tempting, and it costs nothing. But the correlation was measured under the attacks tried so far. An adaptive attacker who knows about the penalty can optimize triggers that erase memory without producing the spike. We include the norm penalty, and a norm-threshold detector, as additional baselines under adaptive attacks that are told about them. If either holds up against those attacks, it is the better defense and we will say so.

Why not just use a hybrid with attention layers, or attention-based dynamic rectification? The attention layers in Jamba-1.7-Mini did not prevent collapse under optimized triggers, and its prompt-injection success rate rose from 49.0% to 74.3% (Hidden State Poisoning Attacks). Rectification methods like Dynamic Attention mute tokens that receive disproportionate attention. They need an explicit $N \times N$ attention map, which a pure SSM never computes, so they are inapplicable to the layers that fail.

Proposed Experiments

The primary evaluation is RoBench-25 accuracy for defended Mamba-1.4b and Mamba-2.8b. We test under the published Z-HiSPA and M-HiSPA triggers, under triggers re-optimized against each defended model, and under trigger lengths two to six times longer than any training window. The last condition tests length transfer in the spirit of the square-root rule. The secondary evaluation is attack success rate on Open-Prompt-Injections, with and without triggers.

Utility is measured four ways: clean RoBench-25 accuracy, perplexity on a held-out slice of The PILE, the synthetic in-context regression and classification suites from Is attention required for ICL?, and the legitimate-reset test. Cost is measured in GPU-hours and peak memory per training step at context lengths of 1K, 4K and 16K. At the same lengths we time discrete-trigger training on Mamba, and adversarial training on Pythia-2.8b, to show how cost scales with length for each approach.

We treat three results as decisive. First, if gate-space training does not reduce the RoBench-25 drop under adaptive token triggers to well below the undefended 71%, the relaxation does not transfer and the core claim fails. Second, if the full-state ablation matches gate-space training, the mechanism-matching story is wrong and the contribution shrinks to generic latent adversarial training for SSMs. Third, if gradient alignment inside the perturbation ball collapses during single-step training, as it does in Understanding and Improving Fast Adversarial Training, while robustness to the training attack stays high, the defense is overfitting to its own attack. In that case we report the smallest number of attack steps that avoids it.