When a language model plays Isaac Newton, it should be unable to explain general relativity. Current models usually explain it anyway. This failure has a recognizable shape. Researchers who study "knowledge conflict" (what a model does when its prompt contradicts facts stored in its weights) have found attention heads and activation directions that decide whether the model trusts the prompt or its memory. Researchers who study role-play have found that characters leak facts from outside their era or world. These two literatures have not been joined. The obvious join is to reuse the existing "trust the context" controls for role-play. We argue that this join is wrong, and wrong in a way that can be tested. A persona profile almost never contradicts the model's knowledge of relativity. It just says nothing about it. Out-of-character leakage is therefore a failure to recognize scope, not a failure of context faithfulness.
We propose to find a separate internal signal for "this fact lies outside the character's world." We read that signal through the model's own next-token head, phrased as a cloze question in the style of prompt-based few-shot learning. We then use it to gate a steering intervention only on the turns where it is needed. If the idea is right, a single "memory versus context" knob cannot reduce anachronistic leakage without also making the character forget things they should know. A two-signal, gated intervention should escape that trade-off without any fine-tuning and without the multi-agent retrieval loops current methods rely on.
Core Claim
Revealing and Mitigating the Challenge of Detecting Character Knowledge Errors in LLM Role-Playing splits character knowledge errors into two kinds. Known knowledge errors (KKE) are misstatements of facts inside the character's scope. Unknown knowledge errors (UKE) are leaks of facts outside it. The paper reports that models are 15–20% worse at catching the first kind. Our claim is that these two errors pull on the parametric-versus-contextual balance in opposite directions.
Catching a KKE about Hermione Granger often requires the model's parametric memory of the books, because the profile is short. Suppressing a UKE requires inhibiting parametric memory that the profile never mentions. The existing mechanistic controls all move the model along one axis. Cutting Off the Head Ends the Conflict prunes late-layer "memory heads" or "context heads" (PH3). ContextFocus adds a steering vector toward contextual faithfulness. On the KKE–UKE plane, those one-axis controls should trace a trade-off curve. We predict that a separate "scope" direction exists in the residual stream, the running hidden state that each layer adds to. That direction should be largely orthogonal to the context-faithfulness direction, and gating along it should move UKE leakage while leaving KKE accuracy and in-scope recall roughly intact.
The third direction in the brief, pre-training task misalignment, explains why this gap exists and also supplies the method. Next-token pre-training mixes every era and fiction into one predictive distribution, so it never trains the judgment "who would know this." The lesson of Making Pre-trained Language Models Better Few-shot Learners is that a new judgment is learned most cheaply when it is phrased as the model's native token prediction, rather than learned through a freshly initialized classifier. We apply that lesson to the gate. We ask the model a cloze question, "In [character's world], [character] had heard of [entity]: yes / no," and read the answer from its own vocabulary distribution. We do not train a probe from scratch.
How This Sits Against Prior Work
Several papers motivate the proposal by documenting the gap. RoleKE-Bench (paper) shows that no tested model exceeds 65% accuracy at detecting character knowledge errors. It also shows that role-play-tuned models are worse at this, and that S²RD, the authors' self-recollection and self-doubt pipeline with Wikipedia retrieval, raises GPT-3.5 from 24.5% to 80.1%. The authors explicitly call for internalizing this ability rather than running costly external loops. We answer that call. We differ from their work in two ways: we evaluate leakage during generation rather than detection of planted errors, and we intervene inside the model.
A second group of papers supplies our mechanistic tools. PH3 locates competing memory and context heads in late attention layers. Competition of Mechanisms shows that factual recall partly works by actively suppressing the competing token. ContextFocus builds steering vectors by contrasting activations under context-reliant and memory-default prompts. All three were developed on short counterfactual swaps (COUNTERFACT, NQ-SWAP, World Capital), where the context states a replacement fact. We reuse their contrastive and path-patching techniques. Our contrast is different: same persona, in-scope entity versus out-of-scope entity, with the context silent on both.
A third group of papers constrains the design. Measuring and Controlling Persona Drift attributes multi-turn drift to declining attention on the system prompt and fixes it with split-softmax. The Assistant Axis attributes drift to movement along a dominant "assistant" direction and fixes it with activation capping. Am I Me or You? attributes it to mirroring the conversation partner. Each of these restores the profile's influence. Our argument predicts that restoring the profile's influence does little for scope, because the profile contains no statement about relativity to restore. That prediction is testable, and we test it. Codifying Character Logic in Role-Playing and Persona-Aware Contrastive Learning improve behavioral fidelity through external code and offline preference training respectively. Neither addresses anachronistic knowledge. We use Codifying Character Logic's Fandom benchmark to check that our intervention does not damage fidelity, and we use PCL's training recipe as a competing baseline.
Research Program
The work has four stages.
Stage 1: a generation-time leakage benchmark. We take RoleKE-Bench's nine characters and its out-of-scope entities, and expand each into open questions where any substantive explanation counts as leakage ("What do you make of the theory of relativity?"). We pair these with matched in-scope questions of similar difficulty. A strong judge model labels each answer as leaked, in-character deflection, or wrong in-scope content, and we validate the judge against a few hundred human labels. For multi-turn testing, we place the same probes at turns 1, 4, 8, and 12 after neutral filler dialogue. The original 990 RoleKE detection items stay as a second, directly comparable metric.
Stage 2: a mechanistic test of the two-axis hypothesis. Using open-weight models of the families studied in The Assistant Axis (Llama-3, Qwen, Gemma), we compute two directions. The first is a ContextFocus-style faithfulness direction from NQ-SWAP/ConFiQA contrasts. The second is a scope direction from same-persona in-scope versus out-of-scope contrasts, measured at the entity tokens and the answer position. We report the cosine similarity between the two directions across layers. We path-patch the heads PH3 identifies to measure how much of UKE leakage flows through them. Finally, we sweep PH3 pruning and ContextFocus strength and plot each sweep on the KKE-accuracy versus UKE-leakage plane. The hypothesis fails if the faithfulness sweep lowers leakage while holding KKE accuracy and in-scope recall steady.
Stage 3: the gated intervention. At each response, a cloze query is formed for the salient entity of the user's message. The query is answered by the frozen model using its native vocabulary head, with verbalizers "yes" and "no," and a few labeled examples are allowed in context. When the "no" probability exceeds a threshold, we subtract the scope direction at the chosen layers in proportion to that confidence. Otherwise we leave the model untouched. The cost is one short additional forward pass per turn.
Stage 4: multi-turn behavior. We test whether scope erodes with turn count. If it does, we test whether that erosion tracks the drop in attention to the profile (in which case split-softmax would combine with our method) or movement along the Assistant Axis (in which case capping would). This links our work to the unresolved debate over what causes persona drift, without making that debate the main bet.
Baselines
The naive reference point is the plain persona prompt with no intervention, run on the same models and decoding settings. A stronger prompt-only reference adds an explicit boundary instruction ("you lived in the 17th century and know nothing after it"), repeated every turn. This is the cheapest reasonable fix, and our method must beat it to justify its added machinery.
The strongest training-free mechanistic competitors are ContextFocus and PH3. For a fair comparison, ContextFocus gets the same number of contrastive examples it reports needing (about 1.5k), and both methods get the same grid search over strength or pruned-head set on a shared development split. Both are scored on the same KKE/UKE plane. Beating them means our method sits on a strictly better Pareto frontier: lower leakage at equal KKE accuracy and in-scope recall, not merely lower leakage at some operating point.
Split-softmax is the strongest competitor for the multi-turn condition. We tune its exponent on the development split and run it alone and combined with our gate.
The strongest competitor on knowledge-error detection is S²RD, which the RoleKE authors report raising GPT-3.5 from 24.5% to 80.1% average accuracy on RoleKE-Bench. We rerun S²RD on our open models with the same retrieval budget it uses and report accuracy alongside wall-clock time and tokens per turn. Matching it within a few points at a small fraction of the cost would count as a win. Exceeding it is not required. For generation-time leakage, no reported result was found on Lacuna for any method, so all leakage numbers are produced in this study.
The strongest training-based competitor is direct preference optimization (DPO, preference fine-tuning from pairs of better and worse responses) in the style of PCL. We train on synthetic pairs that contrast in-character deflection with leaked answers, using the same entity pool our gate's few-shot examples come from and a matched compute budget.
Three ablations isolate the key ingredients. The first removes the gate and applies the scope direction always. The second replaces the cloze gate with a linear probe trained on the same examples. The third substitutes a random direction of matched norm.
Why not just…?
Why not just tell the model what it doesn't know? The boundary instruction is our second baseline. We expect it to help at turn 1 and fade, following the decay Li et al. measured for system-prompt instructions, with GPT-3.5 losing about 10% over extended rounds. A more basic problem is that a sentence like "know nothing after 1700" requires the model to make the same scope judgment it fails at. If this baseline holds leakage flat through turn 12, the mechanistic part of the proposal is unnecessary, and we will report that.
Why not just reuse ContextFocus or PH3? These methods push the model toward whatever the context says. In our setting the context says nothing about the out-of-scope entity, so pushing toward context should either do nothing or suppress parametric knowledge indiscriminately, including the in-scope book knowledge needed to catch KKE. Stage 2's sweep tests this directly. If a one-axis control turns out to reach our frontier, the scope-direction claim is false.
Why not just contrast persona-conditioned logits against a generic prompt at decode time? Subtracting the unconditioned distribution removes whatever the two prompts share. Newton's knowledge of his own laws is shared just as much as his "knowledge" of relativity, so we expect this method to damage in-scope recall. It is a genuine threat on leakage alone, so we include it as a baseline and judge it on the full plane.
Why not just fine-tune with DPO on anachronism refusals? This is the most serious alternative and it may work well. Two pieces of evidence argue against it as a complete answer. RoleKE reports that role-play fine-tuning weakened knowledge-boundary detection, and per-character tuning does not transfer to a new persona without new data. Our method needs only a profile and a few cloze examples. The decisive test uses held-out characters: DPO is trained on six of the nine RoleKE characters and evaluated on the other three. If DPO matches us on the held-out characters, the honest conclusion is that training wins and our contribution is the diagnosis.
Decisive Experiments
The first decisive check is the frontier plot from Stage 2. Faithfulness steering and head pruning should trade KKE against UKE, and the gated scope intervention should dominate both on RoleKE-derived leakage and in-scope recall. The second check is held-out-character transfer against DPO. The third is a side-effect audit. We require no meaningful drop on CharacterEval consistency or on Fandom NLI fidelity, where text profiles score 65.98 and codified profiles 68.38 per Codifying Character Logic. We also check MMLU and ARC, and we confirm that ConFiQA faithfulness is unchanged when no persona is present, which shows the gate stays closed outside role-play. The fourth check is the multi-turn curve: leakage at turns 1 through 12 with and without split-softmax, which tells us whether scope erosion and persona drift share a cause. If the two directions turn out to be nearly parallel in Stage 2, the project changes course. It becomes a negative result showing that existing faithfulness controls already capture scope, measured with a new generation-time benchmark.
References
- Revealing and Mitigating the Challenge of Detecting Character Knowledge Errors in LLM Role-Playing
- Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models
- ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models
- Making Pre-trained Language Models Better Few-shot Learners
- Revealing the Challenge of Detecting Character Knowledge Errors in LLM Role-Playing
- Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
- Measuring and Controlling Persona Drift in Language Model Dialogs
- The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models
- Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identity
- Codifying Character Logic in Role-Playing
- Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learning