Safety training for language models almost always shows the model a clean conversation and asks it to refuse at the right moment. Real conversations are not clean. When a multi-turn jailbreak partly succeeds, the model's own earlier replies already contain partial compliance: a recipe outline, an agreed-upon "fictional" framing, an accepted false premise. Models treat their own past words as settled facts and stay consistent with them. This proposal trains a model to notice that one of its own earlier turns was a mistake and to explicitly take it back, instead of continuing or issuing a generic refusal. Training uses contrastive preference pairs built on conversations the model itself has already compromised. Each harmful case is paired with a lookalike benign conversation in which continuing is the correct behavior.
The opportunity comes from two findings that pull against each other. The self-correction literature finds that models generally cannot spot their own errors without outside help (When Can LLMs Actually Correct Their Own Mistakes?). Backtracking work finds that a model can be trained to recognize an unsafe prefix shortly after producing it and reset (Backtracking Improves Generation Safety). We bet that this trained recognition can be extended across dialogue turns. If that holds, a model can recover in the middle of a conversation without an external judge at test time. If it fails, we will have mapped where intrinsic recovery breaks down as compromised history accumulates.
Core Claim
Current multi-turn safety defenses are trained and evaluated on conversation histories in which the assistant behaved well up to the harmful pivot. Red Queen Guard applies preference tuning (direct preference optimization, DPO, which raises the likelihood of a preferred response over a rejected one for the same context) to multi-turn persona jailbreaks. It reports reducing attack success to below 1% on the Red Queen dataset (Red Queen). MTSA penalizes earlier turns that lead to a final jailbreak, and reports Zephyr-7B attack success dropping from 74.50% to 23.50% at a 5.62% increase in XSTest over-rejection (MTSA). Both methods teach the model not to enter a bad trajectory. Neither teaches it to leave one it has already entered.
That gap matters because dialogue systems that look strong on gold histories degrade badly once they condition on their own outputs (Is Your Goal-Oriented Dialog Model Performing Really Well?). Appended history also turns early mistakes into premises the model will not contradict (LLM-as-RNN; Hallucination Snowballing in LVLMs).
Our claim has two parts. First, defenses trained on clean histories lose effectiveness as the number of self-compromised assistant turns grows. Second, a model trained contrastively to retract on compromised histories, and to continue on matched benign histories, keeps its recovery rate roughly flat in that number. It should do so without raising false retractions on benign but sensitive-looking conversations. The new object of study is recovery from the model's own past compliance. We will measure recovery, relapse, and false retraction, quantities no paper in the file reports. We found no reported result on Lacuna for recovery rates of any defense when conditioned on self-compromised dialogue histories.
Research Program
The core artifact is a compromised-history dataset built from Red Queen scenarios, which cover 14 harm categories and 40 persona-and-pivot scenarios. We add MTSA-style "salami-slicing" decompositions, where a harmful request is split into innocuous-looking pieces. For each scenario we roll out the target model itself under attack and keep transcripts in which some assistant turn $k$ already contains harmful content. An offline judge locates $k$, much as the corrective-intervention work uses GPT-4o to locate compliance cues (Towards Safe Reasoning via Corrective Intervention). The judge is used only to build training data, never at inference. We then truncate at a later user turn $t$, controlling the depth $d = t - k$ from 0 to 4.
At each truncation point, the rejected response continues the compromised trajectory. The chosen response is a recovery: it names the earlier content as something it should not have provided, declines to extend it, and offers whatever legitimate help remains. Recoveries are generated from the model's own distribution by inserting a short corrective opening, following the safety-trigger splicing of the corrective-intervention paper. This keeps the chosen responses close to the model's natural style rather than in a teacher's voice.
The second half of the dataset prevents the obvious shortcut, "retract whenever history looks edgy." Following the four-way contrast of WildTeaming and the benign contrast set of COCONOT, we build a benign twin for every compromised history. The twin keeps the same persona, the same sensitive vocabulary, and the same fictional framing, but its earlier assistant turns are actually harmless, so continuing is the correct choice and retraction is the rejected response.
We also add the contextual-noise setting. Into both arms we insert irrelevant retrieved passages and misleading-but-harmless factual snippets in the style of SciAux (Thinking in a Crowd). Noise alone should therefore never trigger retraction. Without it, the model could learn to retract on any messy context.
We train with DPO on Llama-3-8B-Instruct and Zephyr-7B, which allows direct comparison with Backtracking and MTSA. We also train DeepSeek-R1-Llama-8B to test whether visible step-by-step reasoning helps recovery or rationalizes the earlier compliance. Thinking in a Crowd showed that reasoning can turn misleading context into elaborate justifications. Qwen3-8B with reasoning enabled fell to 77.03% under misleading snippets, from a 96.22% clean baseline.
One structural variant is tested as an ablation. The model may emit a dialogue-level retraction marker, analogous to Backtracking's single-generation reset token, and the serving system then drops the retracted turn from context in later turns. This is the cross-turn pruning that Backtracking explicitly does not support. It tests whether textual retraction alone prevents relapse or whether the compromised tokens must physically leave the context.
Baselines
The naive reference is the instruction-tuned model with no further training. It is evaluated on the same compromised histories and benign twins, with and without a safety system prompt. COCONOT reports that system prompts cause over-refusal while still missing subtle cases, so both the safety and the over-refusal cost of this baseline need measuring.
The strongest existing competitor is Red Queen Guard, meaning multi-turn DPO on clean histories ending at the pivot. It reports below-1% attack success on its own benchmark, the best multi-turn number in the file. It is also the closest method, so if clean-history training already generalizes to compromised histories, our proposal is unnecessary. We reproduce it on the same base models, with the same number of preference pairs and the same scenarios. We match training steps and hyperparameter search budget. Beating it means significantly higher recovery rate at depth $d \ge 1$, at equal or lower false-retraction rate on benign twins and equal XSTest and MT-Bench scores. Matching it only at $d=0$ is expected and does not count.
MTSA's future-reward training is the second strong competitor because it explicitly propagates safety signal across turns. Its reported numbers are 74.50% to 23.50% attack success on Zephyr-7B and +5.62% XSTest over-rejection. We will run it on Zephyr-7B at matched rollout compute rather than its full three-iteration budget, and also at its full budget with that budget reported.
Backtracking's single-generation reset token reports Llama-3-8B violations dropping from 6.1% to 1.5% and GCG attack success from 62.5% to 16.5%. It shows whether within-turn undo already provides most of the benefit.
A pipeline baseline uses an external judge to check each assistant turn and truncate flagged turns from history before the next generation. This is in the spirit of the critic-driven corrective-intervention method and of ACR's external context refactoring (ACR). It is scored on the same metrics plus judge calls per conversation.
Two ablations isolate the key ingredient. One removes the benign twins, keeping only compromised histories, and tests whether the contrast is what prevents over-refusal. The other trains on the same recovery responses but with clean histories substituted for the compromised ones. It tests whether conditioning on the model's own prior compliance is what generalizes.
Why not just…?
Why not just use Red Queen Guard, which already gets attack success below 1%? That figure is measured where the assistant has not yet complied. Our hypothesis is that it holds at depth zero and decays as compromised turns accumulate, because the model learns "refuse at the pivot" rather than "leave a bad trajectory." This is a real threat. If the depth curve for Red Queen Guard is flat, the proposal collapses to a data-augmentation note, and we will report that plainly. The clean-history ablation settles the question with everything else held fixed.
Why not just prompt the model to review the conversation and fix mistakes each turn? The critical survey of self-correction finds that under fair, best-effort initial prompts, intrinsic self-review gives no gain and often flips correct answers to incorrect ones. Thinking in a Crowd shows that extended reasoning over misleading context makes things worse. We include this prompt as a baseline and expect it to raise false retractions on benign twins more than it raises recovery.
Why not just run an external guard model and delete flagged turns? This may well work for safety, and it is the most serious practical alternative. Its costs are an extra model call per turn and the fact that the user-facing model still never acknowledges what it said. Its known weakness is that judges trained on single-turn harm see salami-sliced pieces as harmless in isolation, which is MTSA's central observation. We will report whether it matches our method and at what call cost. If it dominates on both safety and over-refusal, the conclusion is that recovery belongs outside the model.
Why not just refuse whenever the history contains sensitive content? That is the shortcut the benign twins measure directly. COCONOT and XSTest-style results show such rules over-refuse heavily.
Proposed Experiments
The headline experiment plots recovery rate against compromise depth $d$ for every method on held-out Red Queen scenarios and held-out MTSA-style decompositions. Recovery rate is the fraction of compromised histories where the next response retracts and does not extend harm, judged by the HarmBench classifier and a GPT-4o judge with human audit of a sample. On the same plot we show false-retraction rate on benign twins.
The second experiment measures relapse. After a recovery, the attacker model continues for three more turns, and we count re-compliance with and without the context-pruning variant. This tests whether the compromised tokens' pull persists after a textual retraction.
The third experiment adds SciAux-style misleading snippets and irrelevant passages to both arms. It checks that noise neither triggers retractions nor suppresses needed ones. It also compares reasoning on versus off for DeepSeek-R1-Llama-8B, to see whether visible reasoning rationalizes earlier compliance the way it rationalizes misleading facts.
Utility is tracked on MT-Bench, AlpacaEval, and XSTest, and single-turn safety on WildJailbreak and HarmBench. These guard against the gains being bought with general refusal.
The decisive check is a transfer test that separates a learned retraction skill from memorized safety style. Using SciAux questions, we build multi-turn transcripts in which the model's own earlier turn endorsed a misleading premise. We then measure, without any factual training, whether the safety-trained model is more willing to retract wrong earlier claims than the base model. Following the self-correction survey's warning, we also measure how often it wrongly retracts correct earlier claims. Positive transfer with no rise in wrong retractions would mean the model learned to distrust its own history on evidence. Zero transfer would mean the effect is specific to safety. Both outcomes are informative, and the depth curve remains the main test.
References
- When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs
- Backtracking Improves Generation Safety
- Red Queen: Exposing Latent Multi-Turn Risks in Large Language Models
- MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming
- Is Your Goal-Oriented Dialog Model Performing Really Well? Empirical Analysis of System-wise Evaluation
- LLM-as-RNN: A Recurrent Language Model for Memory Updates and Sequence Prediction
- Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models
- Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention
- WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models
- The Art of Saying No: Contextual Noncompliance in Language Models
- Thinking in a Crowd: How Auxiliary Information Shapes LLM Reasoning
- ACR: Adaptive Context Refactoring via Context Refactoring Operators for Multi-Turn Dialogue