Large language models are usually treated as if their weights are frozen and pristine while they chat. They are not. Bit-flip attacks show that an adversary who can disturb memory — electrically, via RowHammer-style leakage, or through quantization rounding — can flip a handful of bits among billions of parameters and jailbreak a production model without touching a single prompt character. PrisonBreak: Jailbreaking Large Language Models with Fewer Than Twenty-Five Targeted Bit-flips needs fewer than 25 bits, and as few as 5, to reach 80–98% jailbreak success across ten model families while general-task accuracy drops only 2.3% on TinyBenchmarks — and it demonstrated a physical RowHammer exploit on an NVIDIA A6000 using as few as two bit locations. Meanwhile, a separate literature shows that safety guardrails erode on their own as conversations get longer: refusal boundaries decay when malicious intent is split across turns, and attack success climbs sharply from 2 to 6 conversational turns. These two failure modes have never been made to interact. Every bit-flip study evaluates single-turn prompts; every multi-turn safety study assumes perfect weights.
The core claim of this proposal is that these failures are multiplicative, not additive. A sparse weight corruption that is individually harmless on single-turn benchmarks — a sub-threshold flip — should become an effective jailbreak once the model is placed in a multi-turn conversation that has already diluted its refusal behavior. If true, the practical picture of hardware-side risk is far worse than either literature suggests alone: an attacker needs fewer flips (cheaper RowHammer exploits), the attack is harder to detect (standard single-turn audits pass), and the affected deployment regime — long agentic dialogues — is exactly where models increasingly live. We will measure this amplification, localize its mechanism, and build a low-overhead defense that follows the cheap-audit philosophy of Alignment Collapse Under KV Cache Quantization: A 35-Minute Audit for Quantized LLM Deployments rather than the expensive iterative-red-teaming philosophy of MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming.
Core Claim
Define $B_1$ as the minimum number of bit flips needed to reach a target single-turn jailbreak success rate (say 50% on AdvBench), and $B_t$ as the minimum needed to reach the same rate in a $t$-turn conversation. We hypothesize an amplification factor
$$A(t) = B_1 / B_t > 1, \quad \text{growing with } t,$$
and, more importantly, a sub-threshold regime: flip configurations whose single-turn jailbreak success is statistically indistinguishable from the clean model, but whose multi-turn success rate rises well above the clean model's own multi-turn baseline. The mechanism is a conjunction: bit flips in the value-projection layers — PrisonBreak found exponent bits of attn.v weights disproportionately vulnerable, and found that corrupted models keep refusal features intact in representation space while the generation head reroutes toward harmful continuations — should combine with the history-dependent erosion documented by multi-turn safety work such as MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety and the attention-dilution account in A Prompt-Masked Pilot for History-Dependent Safety Degradation in Multi-Turn Conversational Agents. Neither the corruption nor the context alone suffices; together they do. This is an AND-gate attack, and AND-gates are the worst case for audits that test one side at a time.
There is a mechanistic wrinkle we must confront rather than ignore, because the literature disagrees with itself about where safety lives. PrisonBreak implicates value projections; the KV-cache quantization study PCR finds that quantizing Keys destroys nearly all alignment in 8 of 9 models (Mistral-7B loses 15.2% of refusals at only a 1.03× perplexity increase, and Qwen-2.5-7B flips 30.3% of refusals under standard FP8 in vLLM) while Values stay resilient to uniform quantization noise; and Securing Multi-turn Conversational Language Models From Distributed Backdoor Attacks locates multi-turn backdoor shortcuts in the final layers. Value projections may be sensitive to sparse outliers (bit flips) but resilient to diffuse noise (quantization), which would be a genuinely useful architectural fact. Our mechanism experiments will test all three loci under both corruption types across turn depths, rather than assuming any one.
Research Program
Phase 1 is the measurement. We run PrisonBreak's progressive bit search (adapting its search objective from a single prompt to a trajectory-level reward, since the original optimizes per-prompt) on Llama-3.1-8B-Instruct, Mistral-7B, and Qwen-2.5-7B, in BF16 and INT8, and record $B_t$ for $t = 1, 2, 4, 6$ turns. The multi-turn attack substrates come from MultiBreak's 10,389 prompts spanning 2–6 turns and its 26 categories; single-turn reference points come from AdvBench. We also run the reverse direction: apply fixed flip budgets drawn from SBFA-style sneaky flips that preserve clean utility (its single bit drops MMLU to 0.00, but we deliberately stop short of that, selecting flips that leave MMLU and GSM8K within noise) and measure whether MultiBreak success rates climb. Success is judged with the HarmBench behavior classifier, whose Llama-2-13B variant reports 93.19% agreement with human labels per HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal.
Phase 2 is the mechanism. Using activation differencing between clean and corrupted models across turn depth, we test three competing hypotheses about locus (value projections, key projections, final layers) and ask whether the corrupted model's refusal signal survives in representation space while failing at generation — PrisonBreak's single-turn finding — or whether multi-turn context actively suppresses the representation itself, as the attention-dilution account predicts. The discriminating experiment: apply DCD-style contrastive decoding between intermediate and final layers and see whether it restores refusal. If refusal features are intact, contrast should help; if context has suppressed them, it should not.
Phase 3 is the defense, and it is deliberately cheap because the fourth research direction — the computational cost of adversarial training — is our binding constraint, not an afterthought. Full iterative red-teaming defenses like MTSA require multi-round RL rollouts; MTSA itself needed three iterations to push Zephyr-7B from 74.50% to 23.50% attack success. We instead build two complementary low-overhead components. The first is a trajectory probe audit: PCR's diagnostic runs in roughly 35 GPU-minutes with 20 prompts; we extend its idea with a fixed bank of short multi-turn probe conversations designed to be maximally sensitive to sub-threshold corruption, catching the AND-gate that single-turn audits structurally miss. The second is corruption-hardened safety tokens: following Robustness Tokens: Towards Adversarial Robustness of Transformers, which froze a DINOv2-Giant backbone and trained only ~10 appended learnable vectors to convergence in 200 steps, recovering white-box ImageNet accuracy from 0.3% to 63.1% at negligible clean-accuracy cost (85.2% vs 85.6%), we train a small set of tokens appended at each conversational turn with a dual objective: refuse consistently under simulated bit flips (sampled from the Phase 1 vulnerability map) and under multi-turn jailbreak rollouts. The backbone stays frozen, so the training cost is a rounding error next to MTSA-style pipelines — and, critically, the defended weights cannot themselves be re-corrupted in the trained component, because the frozen backbone is where flips land and the tokens are re-loadable, verifiable state.
As a secondary thread, we test whether bit flips can install a hardware-distributed multi-turn backdoor: flips that produce compliance only when a specific $k$-of-$n$ combination of benign-looking turns appears. This extends Securing Multi-turn Conversational Language Models From Distributed Backdoor Attacks, whose POISONSHARE installs distributed triggers via 5% data poisoning achieving over 99% attack success, and TBT: Targeted Neural Network Attack With Bit Trojan, which wires a single-input trigger into dormant weights via gradient-ranked flips. Nobody has combined the two injection routes, and a flip-installed distributed trigger bypasses every input-level sanitizer because the "trigger" is the conversation's own benign content.
Proposed Experiments
The headline experiment measures $A(t)$ and the sub-threshold regime on three models and two precisions, reporting bits-to-jailbreak against turn depth alongside utility (MMLU, GSM8K — where SilentStriker: Toward Stealthy Bit-Flip Attacks on Large Language Models shows 50 flips drop GSM8K from 65.7% to 7.6% while keeping text fluent, naturalness 61.1 vs 66.0, a useful stealth reference). Utility and over-refusal checks use MT-Bench, XSTest, and AlpacaEval, since MTSA reports only a 5.62% XSTest over-rejection increase and we must not do worse. The defense evaluation runs the Phase 1 attack suite against the probe audit (detection rate, GPU-minutes, false alarms on benignly retrained checkpoints) and against the safety tokens (multi-turn attack success under fixed flip budgets, with and without tokens, utility deltas). The backdoor thread reports $k$-of-$n$ activation curves. For absolute multi-turn baselines we will measure everything ourselves; MultiBreak reports its ASR as up to 54.0% higher than the next-best dataset on DeepSeek-R1-7B (and 34.6% higher on GPT-4.1-mini), but no absolute clean-model ASR for our target models is reported in the file, so we state: no reported result found on Lacuna for absolute MultiBreak attack success rates on Mistral-7B or Llama-3.1-8B.
Baselines
The naive reference point is the pristine, undefended model: no corruption, no defense, measured on MultiBreak to establish the natural multi-turn degradation floor and on AdvBench for the single-turn floor. Any proposed defense must first be judged against the uncomfortable fact that some jailbreak success exists before anyone touches the hardware. The strongest attack-side competitor is PrisonBreak itself, which holds the published record for bit-efficiency (80–98% ASR with fewer than 25 bits, 2.3% TinyBenchmarks utility drop, and a demonstrated physical exploit using two bit locations); we run its search with its own budget and hyperparameters on our models, evaluating it on both single-turn prompts and the identical MultiBreak trajectories our method uses, so the comparison isolates the trajectory-level search objective and the turn-depth dimension rather than compute or tuning. Beating it means achieving equal or higher multi-turn ASR with strictly fewer flips — or, from the defense side, restoring refusal under its best multi-threaded attack at lower cost than any retraining competitor. The strongest defense-side competitor is MTSA, whose published numbers (Zephyr-7B from 74.50% to 23.50% ASR over three red-teaming iterations, with the red-team model itself reaching 63.92% average ASR against GPT-4o and Claude 3.5) set the bar for attack-success reduction; we will run MTSA on one target model with its published procedure, give it its full rollout budget, and compare ASR reduction per GPU-hour against our frozen-backbone token defense — we expect to lose on raw ASR reduction and win decisively on cost, and we will report both honestly. A second defense baseline is DCD, the training-free contrastive decoding defense from the POISONSHARE paper (reducing Mistral-7B ASR from 89% to 0.35% against its own poisoning backdoor), run verbatim at inference time; this is the fair representative of the "do nothing to the weights" family. Finally, the key ablation isolates our central ingredient: safety tokens trained on clean multi-turn rollouts only, versus tokens trained with simulated bit flips injected during training. If the corruption simulation adds nothing, the AND-gate claim is wrong or the tokens are robust by accident, and either finding matters.
Why not just…?
Why not just run PrisonBreak on multi-turn prompts? Because concatenating an existing attack onto longer conversations is exactly what our Phase 1 does — and it is necessary but not sufficient. PrisonBreak's search optimizes against a single-prompt objective; a trajectory-level jailbreak needs a different reward (success at the final turn of a dialogue), and the interesting scientific quantity, the amplification factor $A(t)$ and the sub-threshold regime, requires controlled dose-response curves across turn depth that no existing paper reports. The attack extension is the instrument, not the contribution; the contribution is the interaction law, the mechanism, and the defense. That said, if Phase 1 finds $A(t) \approx 1$ — no amplification — the defense motivation weakens considerably, and we would report that negative result and pivot to the backdoor thread, which does not depend on amplification.
Why not just adversarially train with MTSA and be done? Three concrete reasons. First, cost: MTSA needs multi-round RL rollouts with future rewards across iterations, the very overhead this proposal is designed to avoid; on our budget we can run it on one model, not three. Second, and more fundamental, it is circular as a defense against parameter corruption: MTSA hardens weights, and hardened weights can be flipped afterward — PrisonBreak found RLHF-aligned models need more flips but are still broken, so post-training corruption sits outside its threat model entirely. Third, it does nothing for already-deployed frozen checkpoints. Our probe audit works on deployed models without any retraining, which is the regime where bit flips actually happen.
Why not just use DCD contrastive decoding at inference? This is a real threat to our defense component and we treat it as such. DCD is training-free, cheap, and reports the largest single defense number in the file (89% to 0.35% on Mistral-7B). But its mechanism assumes the malicious shortcut manifests in final layers while intermediate layers stay clean; PrisonBreak's mechanistic finding points the other way — refusal features remain intact in representation space and the failure is in the generation head's routing, and the corruption site is value projections, not final layers. Whether layer-contrast can see a value-projection flip at turn four is precisely our Phase 2 discriminating experiment. If DCD restores refusal under our attack as well as it does under POISONSHARE, the safety-token component loses its justification and we would keep only the probe audit; we will run that comparison head-to-head with identical trajectories and say which wins.
Decisive Checks
The proposal stands or falls on three numbers. First, $A(t)$: if the flip budget does not shrink with turn depth, the amplification claim is dead, and we will say so. Second, the sub-threshold regime: flips invisible to a single-turn audit (AdvBench ASR unchanged, MMLU/GSM8K within noise) that raise MultiBreak success materially — this is the result that would change how deployment audits are written, because it shows single-turn audits are structurally blind to the realistic threat. Third, defense cost-effectiveness: probe audit detection of sub-threshold corruption in tens of GPU-minutes (comparable to PCR's ~35-minute audit), and token defense recovering a meaningful fraction of multi-turn refusal at a training cost closer to ROB tokens' 200 steps than to MTSA's iterative rollouts. Each check has a pre-registered failure mode, and the backdoor thread proceeds independently of the amplification outcome, since a flip-installed $k$-of-$n$ trigger is a threat even at $A(t) = 1$.
References
- PrisonBreak: Jailbreaking Large Language Models with Fewer Than Twenty-Five Targeted Bit-flips
- Alignment Collapse Under KV Cache Quantization: A 35-Minute Audit for Quantized LLM Deployments
- MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming
- MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
- A Prompt-Masked Pilot for History-Dependent Safety Degradation in Multi-Turn Conversational Agents
- Securing Multi-turn Conversational Language Models From Distributed Backdoor Attacks
- SBFA: Single Sneaky Bit Flip Attack to Break Large Language Models
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- Robustness Tokens: Towards Adversarial Robustness of Transformers
- TBT: Targeted Neural Network Attack With Bit Trojan
- SilentStriker:Toward Stealthy Bit-Flip Attacks on Large Language Models