Do LLMs Check Whether a Contradicting Fact Was Actually Asserted?

Language models are often handed text that contradicts what they learned in training, for example a retrieved passage claiming the Eiffel Tower is in Rome. Research on this "knowledge conflict" asks when a model trusts the passage and when it trusts its own memory. That research treats every sentence in the passage as an assertion. Real text does not work that way. "If the tower were in Rome, the guidebook would say so," "It is not the case that the tower is in Rome," and "Either the tower is in Rome or the guide is confused" all mention the same false fact, and none of them asserts it. Linguists have long described when material is accessible, meaning available to later sentences, and when it is sealed inside a hypothetical, negated, or closed-off part of the discourse. A faithful reader should follow the passage only when the fact sits in an accessible position, and should otherwise fall back on what it already knows.

We propose to measure whether LLMs make this distinction internally, and to find where they make it. Our hypothesis has two parts. First, the known conflict-detection signal in middle layers ignores scope, so it fires just as strongly for a hypothetical claim as for an asserted one. Second, whatever accessibility information the model has acts later, as a gate on the attention heads that copy answers out of the context. If both hold, two things follow. Existing methods that push models toward "contextual faithfulness" should also make them more gullible to facts that were merely entertained. A better intervention would steer toward the context only when the model's own accessibility readout says the fact was asserted.

Core Claim

The research file shows that two literatures have not met. On the conflict side, residual-stream probes find conflicts in middle layers (Analysing the Residual Stream of Language Models Under Knowledge Conflicts, roughly 90% detection at layer 14 of Llama-3-8B, with the eventual memory-versus-context decision predictable at layers 16–17). Path patching isolates "context heads" and "memory heads" (Cutting Off the Head Ends the Conflict), and steering vectors push frozen models toward the passage (ContextFocus). All of this work uses flat question-and-passage inputs in which every contextual fact counts as asserted. On the discourse side, Meaning Beyond Truth Conditions shows behaviorally that LLMs track accessibility mostly through surface words. Disjunction works when "either" is present and drops to chance without it, and double negation fails without "in fact." That study runs no mechanistic analysis and explicitly calls for one, and its minimal pairs contain no conflict between context and memory.

Our claim is that accessibility is the variable that should govern conflict resolution. We give it an operational test: there should be a direction in the residual stream that encodes whether a contextual fact is accessible, that survives controls for lexical cues, and that causally determines whether context heads deliver the counterfactual answer. Prior work does not pose this question. The conflict papers never vary assertional status. The accessibility paper never looks inside the model and never sets up a conflict with memory.

Research Program

The benchmark. We first build a scoped-conflict benchmark from NQ-Swap substitution pairs, each consisting of a question, the original answer, and a swapped counterfactual answer. Every swapped fact is embedded in matched templates that vary only its discourse status:

  • Asserted. The fact is stated plainly.
  • Negated and double-negated. Double negation restores accessibility.
  • Conditional antecedent. The fact appears inside an "if" clause, the classic sealed scope of Discourse Representation Theory (Rules for Pronominalization; BUILDRS).
  • Disjunction, with and without "either."
  • Right-frontier condition. This uses Webber's constraint that pronouns like "that" can refer only to the most recent open discourse segment and its open ancestors (Discourse Deixis). A passage asserts the counterfactual, then a later sentence says "That claim was later retracted." We reorder segments so that the counterfactual is either on or off the right frontier when "that" appears. The two versions use identical words and differ only in structure.

The right-frontier condition is the most important one. Because no operator word marks it, a model that relies on cue words cannot solve it.

Each item ends with the original question. The gold behavior follows from construction: give the swapped answer when the fact is accessible and asserted, and the parametric answer otherwise. For disjunction, the gold behavior is to withhold commitment. The literature notes that humans sometimes allow "telescoping," a flexible reference into a quantifier's scope that formal rules forbid. Because of this, a human annotation pass on a subset will confirm that people actually assign the intended status before any condition is scored.

The main measure. Let $A_s$ be the rate at which the model prefers the swapped answer over the original answer in condition $s$, measured by comparing log-probabilities. The accessibility gap is

$$G = A_{\text{asserted}} - A_{\text{inaccessible}},$$

where "inaccessible" pools the sealed conditions. An ideal reader has $G$ near 1. A model that copies every contextual fact regardless of scope has $G$ near 0.

The mechanistic analysis has three stages.

  1. Detection probe. We rerun the conflict-detection probe from Analysing the Residual Stream across conditions. We expect it to fire equally for asserted and sealed facts, which would mean detection ignores scope.
  2. Accessibility probe. We train a linear accessibility probe at the counterfactual entity's token positions and at the final position. We then test whether it transfers to the right-frontier and no-"either" conditions after training only on cue-marked conditions. This transfer test separates a structural variable from a detector of words like "if" and "not."
  3. Context heads and causal patching. We find context heads with the path-patching procedure of Cutting Off the Head Ends the Conflict and measure whether their attention to the counterfactual token drops in sealed conditions. We then patch the accessibility direction from asserted runs into sealed runs and the reverse. If the direction is causal, patching should flip the context heads' contribution and the final answer. This is the decisive mechanistic test.

The intervention. We build a gated version of ContextFocus. The steering vector is added only when the accessibility readout at the counterfactual span exceeds a threshold. We compare it with ungated steering on both our benchmark and plain NQ-Swap.

Models. We use Llama-3-8B base and instruct, which match the residual-stream study and the Llama-3 family in the accessibility study, and Qwen-2.5-7B as a check on a second model family (Unveiling Knowledge Utilization Mechanisms studies both families).

Baselines

Reference points.

  • Do nothing. The naive reference is the unsteered frozen model reading the same items. It gives the raw accessibility gap, and every other method is scored relative to it.
  • Lexical-cue control. A logistic regression on bag-of-words features of the passage, with access to the same training split as the accessibility probe. This is the ablation that matters most for the interpretability claim. A residual-stream probe counts as structural only if it beats this control on the right-frontier and no-"either" conditions. The control should be near chance there because those conditions are lexically matched by design.
  • Random direction. A random vector of matched norm, patched in place of the accessibility direction. This rules out generic disruption as the cause of answer flips.

Existing faithfulness methods. The strongest competitors on the faithfulness side are ContextFocus and PH3.

  • ContextFocus (ContextFocus) extracts a steering vector by contrasting runs with context and instructions against unguided runs, then adds it to middle layers. We use its 1.5k NQ-SWAP examples for extraction and its layer and multiplier choices. Ungated ContextFocus is also the direct ablation for our gating ingredient.
  • PH3 prunes memory heads to raise context use. It reports +38.5% external-context usage and +6.2% recall on NQ.

The file gives no ConFiQA numbers for ContextFocus, and no reported result exists on Lacuna for any method on scoped or embedded counterfactuals. Our prediction is that both methods raise $A_{\text{asserted}}$ and also raise $A_{\text{inaccessible}}$, which shrinks $G$. Our gated method beats them if it achieves a larger $G$ than both, and keeps their NQ-Swap faithfulness within two points of ungated steering.

Decoding and symbolic comparisons.

  • Adaptive Contrastive Decoding (Adaptive Contrastive Decoding) weights context by token-level entropy. It reports 42.06 exact match against 24.75 for plain context-aware decoding on noisy TriviaQA with Llama-2-7B. We run it with matched prompts.
  • Symbolic tagger. A tagger strips sealed clauses using our construction labels. It serves as an oracle upper bound, not a competitor, because it knows the templates.

Fairness rules. All methods share the same items, the same models, the same answer-scoring rule, and an equal hyperparameter search budget on a held-out development split.

Why not just…?

Why not just instruct the model to use only facts the passage asserts as true, or ask it to reason step by step? This is the cheapest fix and we run it. There is prior evidence against it. Meaning Beyond Truth Conditions found that direct prompting of GPT-4o-class models still collapses on disjunction without "either" and on double negation. Note-style reasoning (Chain-of-Note) raised exact match by 7.9 under irrelevant retrieval, but it raised latency from 0.6 to 12.0 seconds per example. If instruction plus reasoning closes the gap on the right-frontier condition, our intervention is unnecessary as engineering. The mechanistic question would still stand: does the model represent accessibility at all when it is not reasoning out loud?

Why not just parse the passage and delete hypothetical or negated clauses before the model sees it? On our templated items this would work, which is exactly why it appears as an oracle and not as a rival. Its weakness is coverage. Classical systems that do this (Automatic Construction of a Knowledge Base) run on narrow grammar fragments. Right-frontier judgments require a discourse tree that no robust parser supplies. We do not claim the model beats the oracle. We claim the model's own readout recovers most of the oracle's gap without an external parser, and we report the remaining difference honestly.

Why not just use contrastive decoding, which already adapts how much to trust the context? Adaptive Contrastive Decoding needs two forward passes per token. It also decides trust from the confidence of the predicted answer, and a fluently phrased hypothetical can produce a confident answer. If it nonetheless achieves a large accessibility gap, that is a real threat, and the head-to-head on $G$ settles it directly.

Why not just fine-tune on scoped-conflict data? Fine-tuning would probably raise $G$ on the trained templates, which makes it a real threat as an engineering answer. It would not tell us whether the model already represents accessibility. We include a small LoRA baseline trained on half the conditions and test it on held-out conditions, especially the right frontier. Failure to transfer would indicate it learned the templates, not the structure.

Proposed Experiments and Decisive Checks

Experiment 1: behavior. We measure $A_s$ and $G$ per condition for all models and baselines. Two predictions follow from the accessibility study. Gaps should be large for cue-marked conditions. They should be near zero for the right-frontier and no-"either" conditions.

Experiment 2: does detection ignore scope? We compare detection-probe accuracy between asserted and sealed items. Similar accuracy supports the claim that detection ignores scope. A large difference would mean accessibility is resolved before or during detection, and we would revise the gating story.

Experiment 3: the decisive mechanistic test. We train the accessibility probe on cue-marked conditions and test it on the right frontier, then run the patching experiment. Three outcomes are possible:

  • The probe transfers above the lexical control, and patching it flips context-head output and answers. This would show that LLMs carry a structural accessibility variable that gates the copying of contextual facts.
  • The probe works only where cue words exist. This would extend the cue-word finding of the accessibility study from behavior to internal representations, with the specific layers identified.
  • The probe transfers but patching does nothing. This would mean the information is present but unused, which explains the behavioral failures differently.

Each outcome is publishable and distinguishable.

Experiment 4: costs of faithfulness methods. We test whether ungated ContextFocus and PH3 reduce $G$. Then we test whether gating recovers $G$ while keeping NQ-Swap faithfulness. This connects the mechanism to the gullibility problem that the conflict literature describes as unsolved.