A language model answering a multi-step question from retrieved documents faces a quiet failure. At some step, a document asserts a fact that contradicts what the model learned in pre-training. This is a knowledge conflict between the model's stored, or parametric, knowledge and the contextual knowledge in its prompt. The model can silently side with memory at that one step, and because every later step conditions on what came before, the error carries into the final answer. Current fixes treat the whole generation uniformly. They either steer the model toward the context at every token, prune attention heads for the entire run, or throw the chain away and start over. Scheduling research solved an analogous problem decades ago. When a machine breaks down mid-shift, you do not recompute the factory's whole schedule. You patch only the disrupted tasks and let a short planning window roll forward.
This note proposes to carry that idea into decoding. The model writes its reasoning one step at a time, and each step is committed only after a cheap check of the model's internal activations for a conflict signal. When the check fires, the system rolls back that step alone and re-decodes it with a targeted intervention. It then continues without touching the steps already committed. The core claim is that when and where a conflict intervention is applied matters as much as what the intervention is. Event-triggered, step-local repair should match always-on steering in faithfulness to counterfactual context. It should do less collateral damage on steps and questions that involve no conflict, and it should cost fewer tokens than regenerating the chain.
What Exists and What Is Missing
The motivating evidence comes from interpretability work. Analysing the Residual Stream of Language Models Under Knowledge Conflicts shows that a linear probe on Llama3-8B's residual stream (the running hidden state that each layer adds to) detects conflict with about 90% accuracy at layer 14 on NQSwap, Macnoise and ConflictQA. The same paper shows that whether the model will follow memory or context becomes predictable by layers 16–17, with AUROC above 0.85. That study is purely diagnostic, and it probes only short single-answer prompts. We reuse its probe as a runtime trigger. The open question is whether a probe trained on single-hop QA still fires at the right step inside a multi-hop chain.
The intervention side supplies our repair operators, and it also defines what we must beat. ContextFocus builds a steering vector, a fixed direction added to intermediate-layer activations, from about 1,500 NQ-SWAP examples. It applies the vector with one static strength at every generation step. Cutting Off the Head Ends the Conflict identifies "memory heads" and "context heads" and permanently zeroes the harmful ones (PH3). It reports gains of up to 6.2% on Natural Questions. Both methods act uniformly over a run. Neither asks whether a given step involves a conflict at all.
From Extraction to Deduction is the closest trained competitor. It fine-tunes one model to act as a critic that masks conflicting answer spans and as a reasoner that deduces from the remaining premises. It reports a 25.99% accuracy improvement over SelfRAG and InstructRAG when the context contains wrong answer spans, using 251 generated tokens per query versus 518 for a distilled reasoning model. Its critique happens once, before reasoning, and it needs SFT plus DPO. Our method works on a frozen model and intervenes during reasoning, at the step where the conflict actually arises.
The scheduling lens comes from Reactive Plan Revision, which repairs only the tasks a disruption affects. It also draws on rolling-horizon scheduling (Underground mine scheduling modelled as a flow shop), which commits a short window of decisions and rolls forward as feedback arrives. The hallucination literature explains why local repair matters for language models. An early wrong token becomes context the model treats as true (Samuel Holt et al.). Catching it before it is committed is therefore cheaper than cleaning up after.
Core Claim
Consider a frozen model that writes its reasoning as delimited steps and checks the layer-14 conflict probe at each step's answer-bearing tokens. When the probe fires, the model rolls back that step by truncating its attention cache and re-decodes it with steering. Its strength is scaled by probe confidence, and all other steps stay unsteered. We claim this procedure will reach the context faithfulness of static ContextFocus steering on multi-hop counterfactual questions. We further claim it will lose less accuracy on questions with clean, non-conflicting evidence. Finally, we claim it will use substantially fewer tokens than regenerating the whole chain under steering once a conflict is detected.
The prediction that separates this claim from the alternatives concerns collateral damage. Static steering perturbs bridging steps, arithmetic and formatting tokens that never involved a conflict. Event-triggered steering should not.
Research Program
The system has three parts. Each is built from existing components.
The trigger. This is the residual-stream conflict probe. We retrain it as in the residual-stream paper on Llama3-8B with NQSwap, then evaluate it per step, without retraining, on ConFiQA. ConFiQA's multi-hop splits (MR, with one counterfactual per chain, and MC, with several) record which hop carries the substituted fact. That gives us step-level ground truth for when a trigger should fire. The decision probe at layers 16–17 serves a second role. It predicts whether the model is about to side with memory, so we fire only when conflict is detected and the model is predicted to go the wrong way.
The repair operator. It re-decodes only the flagged step. The default operator applies a ContextFocus-style vector with strength proportional to probe confidence.
In-context learning enters here as an alternative operator rather than as the core mechanism. Its role is to test whether a demonstration-derived signal can substitute for a trained steering vector. We compare two variants. The first inserts a few conflict-resolution demonstrations as text before re-decoding the step. The second compresses those demonstrations into a task vector and patches it in without the text. In-Context Learning Creates Task Vectors showed that such a vector, taken from around the middle layers, recovers 80–90% of few-shot accuracy on single-token tasks. It also showed that the vector overrides conflicting text instructions. Whether that holds for a reasoning step rather than a single token is itself an open question this arm answers.
The commit horizon. This is the scheduling knob. With a one-step horizon, every step is checked before it is committed. With a horizon of $k$ steps, the probe is read only every $k$ steps, and a trigger rolls back the whole window. Larger windows mean fewer checks and cheaper decoding. They also let the model see a conflicting fact used in a later hop, which may make the conflict easier to detect, at the price of discarding more work on each rollback. We sweep $k$ from 1 to the full chain length. At full length the method becomes "detect, then regenerate everything," which is exactly the regenerate baseline. This lets us trace the full cost–accuracy curve between local patching and global re-solving.
Baselines
Unsteered chain-of-thought. The do-nothing reference is the same Llama3-8B writing step-by-step reasoning over the same retrieved documents, with the same prompt and decoding settings. It sets the floor for faithfulness on ConFiQA and the ceiling for accuracy on clean multi-hop questions. Any intervention must be judged against both.
Static ContextFocus steering. This is the strongest training-free competitor, because it targets the same failure with the same kind of intervention on a frozen model. We rebuild its vector from the same NQ-SWAP examples used for our probe and apply it at every token. We tune its layer and multiplier on a held-out ConFiQA development split with the same search budget we give our trigger threshold and strength. The file reports only that ContextFocus achieves "significantly higher" substituted-answer rates than unsteered models on ConFiQA. No reported numeric result was found on Lacuna for ContextFocus on ConFiQA, so we will reproduce it. Beating it means matching its substituted-answer rate on ConFiQA MC within two points while losing clearly less accuracy on clean HotpotQA and MuSiQue.
Detect-then-regenerate under steering. This is the full-horizon end of our own sweep, stated as a separate baseline because it is what a skeptic would build first. It uses the same trigger, but on any detection the whole chain is regenerated with static steering. It isolates the value of locality. We compare total generated tokens and forward passes at matched accuracy, and we count it beaten if local repair reaches its accuracy with markedly fewer tokens.
Critic–Reasoner. From Extraction to Deduction is the strongest trained method. Its reported gains are 25.99% accuracy over SelfRAG and InstructRAG in adversarial contexts, with 251 tokens per query. We will run their released recipe only if the recipe is available and budget allows. Otherwise we compare against their reported ConFiQA and multi-hop setting, noting that they trained and we do not. We do not expect a frozen model with local steering to beat a model fine-tuned for this task outright. The fair question is how much of the gap a training-free method closes, and at what token cost.
Ablations. Two ablations isolate the key ingredient. The first uses random triggers at the same firing rate as the probe. It tells us whether which steps get repaired matters, or only how many. The second fires on a fixed token schedule, such as always the final hop. It tests whether a crude rule captures most of the benefit.
Why not just…?
Why not just steer all the time, as ContextFocus does? If context should always win, static steering may be enough, and it is simpler. This is a real threat. Our claim rests on the untested premise that always-on steering damages steps that involve no conflict. The clean-context evaluation settles it directly. If static steering loses no accuracy on HotpotQA, 2WikiMHQA and MuSiQue with clean documents, the collateral-damage argument fails. The proposal would then reduce to an efficiency story, or it would not be worth pursuing.
Why not just ask the model in text whether each step contradicts the documents? A Chain-of-Note style self-check (Zhang et al.) needs no probe and no labeled conflict pairs. Evidence from From Extraction to Deduction cuts against it in two ways. Generators show a 4.6x attention gap toward explicit answer strings, and verbal reasoning doubles token cost. Neither point proves a per-step self-check would fail. We therefore run it as an interchangeable trigger inside the same commit-after-check loop, using the same repair operator. We compare detection AUROC against step-level ConFiQA labels, final accuracy, and tokens. If the text check matches the probe, we will report that the scheduling structure, not the internal probe, carries the result.
Why not just fine-tune the model to prefer context? Preference-tuned context priority (Zhang et al.; Song et al.) and the Critic–Reasoner show that training works. Training also fixes one policy for all inputs, and entity-frequency bias resists shallow alignment (Wang et al.). We do not claim to beat trained systems. We claim a frozen-model alternative whose repairs are local and inspectable, and the Critic–Reasoner comparison will show honestly how large the remaining gap is.
Proposed Experiments
Probe transfer (the decisive first check). We measure the layer-14 probe's per-step AUROC on ConFiQA MR and MC, using the labeled counterfactual hop. If AUROC falls below roughly 0.7, the internal trigger does not transfer to reasoning chains. We would then switch the trigger to the text self-check and treat probe failure as a finding in its own right.
Main comparison. We run all baselines on the ConFiQA QA, MR and MC splits, reporting the rate of answers that follow the substituted context fact. We report exact match on clean HotpotQA, 2WikiMHQA and MuSiQue. For every method, we record total generated tokens, rollbacks per question and false-trigger rate.
Horizon sweep. We plot accuracy against tokens as the commit horizon $k$ grows. We expect an interior optimum. A curve that is flat in $k$ would mean locality buys nothing.
Repair-operator arm. We compare the steering vector, inserted demonstrations and the patched task vector on flagged steps only. This tests whether demonstration-derived vectors can stand in for trained steering on multi-token reasoning steps.
Gullibility stress test. Adversarial documents are included in which the context, not memory, is wrong. Here the decision probe's prediction and a fixed "follow context" policy come apart. This exposes the method's main limitation: it knows when to intervene but inherits its choice of which way to push from the task specification.
References
- Analysing the Residual Stream of Language Models Under Knowledge Conflicts
- ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models
- Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models
- From Extraction to Deduction: Resolving Functional Misalignment in RAG via a Collaborative Critic-Reasoner Framework
- Reactive Plan Revision
- Underground mine scheduling modelled as a flow shop: a review of relevant work and future challenges
- Data-Driven Discovery of Dynamical Systems in Pharmacology using Large Language Models
- In-Context Learning Creates Task Vectors
- Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models
- Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning
- Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse
- A Causal View of Entity Bias in (Large) Language Models