Safety That Ignores the Voice of the Conversation but Not Its Content

Chat models fail in long role-play in a specific way. A user casts the model as a forensic chemist, a grizzled mercenary, or a confidant. Over several turns the model talks itself into giving help it would refuse if asked plainly. Interpretability work shows why: across turns, the internal signal that triggers refusal fades, and the persona's name and voice pull the model toward "staying in character." We propose to train a model so that, at every turn of a dialogue, its internal safety decision matches the decision it would make if the same conversation were rewritten in plain, persona-free language. The rewrite keeps everything that was actually requested, accumulated over all previous turns, and removes only the character voice, the fictional framing, and the model's own in-character replies. The constraint applies only to the internal coordinates that carry the refusal decision, not to the words the model outputs, so the model can still refuse, or simply help, in the character's voice.

The bet is that the invariance most defenses implicitly reach for is the wrong one in multi-turn settings. Matching the model's behavior to a persona-free version of the current message is blind to intent that has been split across turns, because the last message usually looks harmless. Matching the model's output tokens to a neutral version strips the style users came for. The right target is narrower: the safety decision should be invariant to the register of the history and fully sensitive to its content. If this works, it gives a training-time defense that needs no extra machinery at inference and that targets the per-turn drift in refusal activations that current work only diagnoses.

Core Claim

Three lines of work motivate the proposal. State-Dependent Safety Failures in Multi-Turn Language Model Interaction builds a role-play attack, STAR, that sets up a role suited to the harmful query and swaps the model's partial refusals for benign surrogate replies. STAR reaches a 94.5% safety failure rate on GPT-4o. Its probes show refusal activations decaying monotonically across turns, with named persona entities acting as anchors. The paper mentions monitoring refusal suppression jointly with role anchoring, but it never builds a defense. A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks runs the Crescendo attack against a circuit-breaker-hardened Llama-3-8B and gets 54.2% attack success. It also finds that probes with 99.9% single-turn accuracy label the final harmful states as benign, because the conversation history keeps activations inside the benign cluster. Both papers are diagnostic, and both locate the failure in the internal state trajectory rather than in any single message.

The closest existing method is Disentangling Intent from Role, which also supplies our core technique. It trains with a one-directional KL penalty that pulls the model's output distribution on a persona-wrapped query toward its output on the same query asked directly, and it never pulls in the reverse direction. This cuts average attack success from 60.1% to 5.4% on Qwen and from 30.2% to 5.2% on Llama-3.1, in single-turn tests. We keep its one-directional teacher-student structure and change two things.

The first change is the teacher's context. Our teacher sees a content-preserving neutral rewrite of the entire history up to turn $t$, not a persona-free single query. This is what lets the teacher catch intent spread across turns.

The second change is where the constraint lands. We apply it to a multi-dimensional refusal subspace, not to the full token distribution, which leaves the student free to keep its style.

The second choice is shaped by negative results in the entanglement literature. Steering Externalities shows that benign steering vectors push harmful queries into benign activation clusters. It also shows that simply projecting those vectors orthogonal to a safety direction fails, because the distortion is non-linear and localized. Separately, Fast Multi-dimensional Refusal Subspaces via RFM-AGOP finds that refusal in recent models such as Qwen 3 8B occupies several directions, and that at least three must be ablated to cross 50% attack success. We therefore anchor to a multi-dimensional refusal subspace and enforce consistency by training rather than by geometric projection at inference.

Concretely, let $h_t$ be the real dialogue history at turn $t$ and $n(h_t)$ its neutral rewrite. Let $a_\ell(\cdot)$ be the model's hidden state at layer $\ell$ for the final position. Let $P_\ell$ be a $k$-column basis of the refusal subspace at that layer, extracted with RFM-AGOP. The invariance term is

$$ \mathcal{L}_{\text{inv}} = \sum_{t}\sum_{\ell \in \mathcal{S}} \big\| P_\ell^\top a_\ell(h_t) - \mathrm{sg}\big[P_\ell^\top a_\ell(n(h_t))\big] \big\|^2 , $$

where $\mathrm{sg}$ stops gradients so the neutral side acts as a fixed teacher, and $\mathcal{S}$ is a small set of middle layers that includes the region where STAR locates the safety bottleneck. This term is added to an ordinary next-token loss on two kinds of targets. Benign turns use in-character reference replies. Harmful turns use an in-character refusal, produced by rewriting the teacher's neutral refusal into the persona's voice.

Research Program

The first component is the neutralizer. This is an instruction-tuned rewriter used only at training time. It removes persona names, fictional framing, and stylistic markers. It rewrites the model's own earlier turns into plain assistant voice, and it restores any refusals that a STAR-style intervention had replaced. It must keep every concrete request and every piece of information already exchanged.

Its fidelity is measured directly. A judge model checks whether the neutral and original transcripts make the same set of requests and contain the same disclosed information. We report the conversations where the neutralizer drops content, because that is the main way this method can quietly fail.

The second component is the training corpus. It has three parts:

  1. Attack trajectories. These come from persona search in the style of the lineage-evolution attack in Disentangling Intent from Role, STAR-style history substitution, and the evolved persona prompts of Enhancing Jailbreak Attacks on LLMs via Persona Prompts. All are extended to 2–12 turns.
  2. MultiBreak sequences. These come from a held-in split of the categories in MultiBreak.
  3. Benign role-play. These are long, persona-rich dialogues of matched length, so the model sees as many in-character turns that should not be refused as ones that should.

The third component is the refusal subspace itself. It is extracted once with RFM-AGOP on the base model. It is then refit periodically during training, because the subspace may move as the model changes. Held-out probes check that the refit subspace still predicts refusal.

The analysis tracks each turn's projection onto this subspace. The aim is to show whether STAR's monotonic decay and the "benign nudge" seen with Crescendo flatten after training, rather than merely that end-of-dialogue attack success drops.

Baselines

Undefended model. The naive reference point is the instruct model with its standard system prompt: Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct, chosen to match the families in Disentangling Intent from Role. We also include the circuit-breaker checkpoint Llama-3-8B-Instruct-RR studied in A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks, where Crescendo reached 54.2% attack success. Any method that does not clearly beat these numbers on our multi-turn suites is not worth pursuing.

Persona-invariant consistency learning, extended to multi-turn data. This is the strongest existing competitor. We run it on our exact multi-turn corpus, with its teacher being the persona-free version of the current user message, and with the same base models, number of training steps, and hyperparameter search budget as our method. Its reported single-turn result is a drop in average attack success from 60.1% to 5.4% on Qwen and from 30.2% to 5.2% on Llama-3.1. We expect it to stay strong on attacks where the final message is overtly harmful. We expect it to fail where intent is distributed across turns and the last message is innocuous, because there its teacher has nothing to refuse. Beating it means two things. First, lower attack success on the decomposed-intent subset of MultiBreak and on STAR-style trajectories. Second, no loss in attack success on single-turn HarmBench, where it is already near 5%.

Safety-contrast steering. We include STEER-BIND from Steering Externalities, which builds the steering vector with safety contrast data and was reported to reduce benchmark attack success from 36% to 5%. We adapt it by building a persona steering vector with safety contrast data. This is the best inference-time representation method in the file.

Prompt-only grounding. We include periodic grounding prompts every three turns, following A Prompt-Masked Pilot, which reported a 0.55-point reduction in confirmation score on its 12-turn cohort.

Decoding-time defense. We include RDS from Root Defense Strategies, with its classifier applied over the full transcript. It reports a 0% false-refusal rate on Xstest for Vicuna-7B, so it sets the bar for over-refusal.

Ablations isolating each ingredient. Two ablations separate the contributions:

  1. Change only the teacher. Use the persona-free last message instead of the neutralized full history, with everything else identical. This isolates the teacher's context.
  2. Change only where the constraint lands. Use one-directional KL on the full output distribution, and separately a one-dimensional refusal direction in the manner of Refusal in Language Models Is Mediated by a Single Direction, instead of the $k$-dimensional subspace. This isolates the subspace choice.

The first ablation tests the content-sensitivity half of the claim. The second tests the style-preservation half and the multi-dimensionality argument.

Why not just…?

Why not just run the existing persona-invariance method on multi-turn data? This is the obvious extension, and it is the main baseline above rather than a straw man. The argument against it is structural. Its teacher is the direct form of the current query. In Crescendo-style and STAR-style dialogues the current query is often benign on its own; the representation-engineering study finds harmfulness signals peak at turn one and fade by turn two. A teacher conditioned on that query will comply and teach the student to comply. The decomposed-intent subset comparison will settle whether this matters in practice. If it shows no gap, our first ingredient is unnecessary and we will say so.

Why not just add a guard model that reads the whole transcript? This is a real threat. A transcript-level classifier can, in principle, see intent spread across turns. A guard that first neutralizes the transcript and then judges it would use exactly our key ingredient at inference. We include this neutralize-then-judge guard as a baseline with the same neutralizer. If it matches our trained model on attack success and false refusals, then our contribution reduces to the neutralization insight, and the case for training rests only on lower inference cost and on producing in-character refusals rather than blocked replies. We will report that outcome honestly. Our expectation that it falls short comes from the over-defense of response-level filters noted in Root Defense Strategies, such as 24% false refusals for SafeDecoding on Xstest, and from the fact that a block is not a reply, so it breaks the role-play.

Why not just periodically remind the model of the safety rules? The prompt-masked pilot shows this helps only partly and leaves a trace. Confirmation scores still spike in turns 5–7. A classifier recovered which safety scaffold was active from register alone with 55.6% accuracy, meaning the reminders change how the model sounds. STAR also shows system-level alignment being overridden as context accumulates. We test periodic grounding directly. We also apply the same register-detection test to every method, so a defense that "works" by making the model sound like a compliance officer is penalized.

Why not just project the persona direction orthogonal to refusal at inference? Steering Externalities tested exactly this projection and found it does not prevent the safety collapse, because the distortion is non-linear. RFM-AGOP adds that a single refusal direction is the wrong object for recent models. Our one-dimensional ablation tests the second point within our own setup.

Proposed Experiments

Models. Training uses Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct. Llama-3-8B-Instruct-RR serves as an additional starting point, to test whether the method repairs the circuit-breaker failure.

Safety evaluation. We use three suites:

  1. Held-out MultiBreak categories. The paper reports only relative gains, such as up to 54.0% higher attack success than prior datasets on DeepSeek-R1-7B. So we establish absolute baseline numbers ourselves.
  2. Crescendo and STAR attacks re-run on open models. These use HarmBench and JailbreakBench goals. STAR's 94.5% figure is on GPT-4o, so open-model numbers must be measured fresh.
  3. A scaled-up version of the prompt-masked delusional cohort. We expand beyond its 27 conversations and use its −1 to 4 rubric, scored with prompts masked.

Style and relevance evaluation. We use four measures:

  1. Role-play quality. CharacterEval and RAIDEN. VeriRole reports 0.8268 average on RAIDEN at 32B, a ceiling reference rather than a like-for-like comparison at our scale.
  2. Persona-conditioned relevance. ConvAI2 Hits@1 and perplexity. PAA reports 93.9% and 14.03 with a purpose-built architecture.
  3. Long benign role-play. A new set of 12-turn benign role-play dialogues, judged for contextual relevance and in-character consistency at each turn.
  4. Over-refusal and general capability. Xstest false-refusal rate and MMLU.

Decisive checks. Five results would decide the question:

  1. Decomposed-intent gap. On trajectories where the final message alone is benign, our method should cut attack success well below the last-message-teacher ablation.
  2. Flattened refusal trajectory. The per-turn projection onto the refusal subspace should stay flat across turns under STAR attacks, where it decays in the base model. Single-turn probes should regain accuracy on multi-turn final states.
  3. No register leak. On benign long role-play, a classifier trained to tell the defended model from the base model should perform near chance. CharacterEval should be statistically indistinguishable from the base model.
  4. No middle-dialogue spike. On the delusional cohort, the turn 5–7 spike should disappear without an increase in Xstest false refusals.
  5. Neutralizer fidelity. Attack success should be reported separately for conversations where the neutralizer dropped requested content. If those dominate the remaining failures, improving the rewriter, not the loss, is the next step.