When a language model answers "yes" to "Does cracking your knuckles cause arthritis?", evaluators conclude it does not know the fact. That conclusion may be wrong. The model may hold the correct fact internally and still output the folk belief, because the human-sounding way the question is phrased pushes it toward what people typically say. This proposal borrows a tool from algorithmic fairness to tell the two cases apart. The tool is path-specific causal effects: a way of splitting the influence of one variable on a decision into the part that flows through an allowed intermediate and the part that goes around it. We treat the misconception-laden framing of a question as the "sensitive attribute." We treat the model's internal factual state as the allowed intermediate. The question is whether falsehoods travel through a corrupted fact or through a direct imitation route that bypasses an intact one.
The distinction matters for two reasons. It answers, per item and with a causal test rather than a behavioral guess, the competence-versus-performance question that comparative-cognition critics say current LLM evaluation cannot answer. It also gives a mechanistic reading of inverse scaling on TruthfulQA. Bigger models could be more wrong because they know less under pressure, or because they imitate better while knowing the same.
Core Claim
We claim that a large share of TruthfulQA errors in 7–13B open models are "knowledge-intact" errors. In these cases the framing of the question changes the answer mostly through a direct path, while an identifiable mid-layer factual state stays close to what the model computes for a neutrally phrased version of the same claim. We further claim that this share grows with model size within a family, which would locate inverse scaling in the imitation path rather than in the knowledge.
The measurement is new. We use Pearl-style nested counterfactuals on activations, with the path partition treated as a normative choice in the fairness sense. We score each error as a competence failure or a performance failure. Mitigation is a secondary, falsifiable use of the same machinery.
The motivating evidence comes from three sources. TruthfulQA built 817 questions that elicit human misconceptions and found larger models less truthful: the best GPT-3 configuration reached 58% against a 94% human baseline. It offered no decomposition of why. Anthropocentric bias and the possibility of artificial cognition names the error of inferring missing competence from behavioral failure "Type-I anthropocentrism" and calls for mechanistic checks, but runs no experiments. Agrawal et al. show that fabricated answers become unstable under indirect questioning, which hints that output and internal belief come apart. They stop at behavioral inconsistency.
The method comes from fairness. Path-Specific Counterfactual Fairness and Kilbertus et al. define allowed "resolving" mediators and nested interventions $Y(a', M(a))$. Shpitser et al. stress that which paths are allowed is a normative decision. All three stay on tabular data with low-dimensional mediators. We move the estimand into a transformer's residual stream.
Causal work on hallucination already exists, and it constrains our framing. Dual-Path Attention Intervention, Dismantling Pathological Shortcuts, Intervene-All-Paths and Treble Counterfactual VLMs define paths as vision-versus-language routing in multimodal decoders and aim to fix outputs. They do not ask whether a failure reflects absent knowledge. RELIANCE uses "counterfactual" only for entity-swap data augmentation.
Research Program
Paired framings. For each TruthfulQA question we build a matched pair. The framed version $a$ is the original question, optionally with a misconception-asserting preamble. The neutral version $a'$ is a verification query about the same proposition, for example "Is the following claim supported by medical evidence: …". The two share a verbatim core span, so token positions over that span align one-to-one. As a control, we build pairs of the same form from HaluEval's HotpotQA questions, which carry no folk misconception (HaluEval).
Outcome. The outcome $Y$ is the log-odds that the model prefers the true over the imitative answer in TruthfulQA's multiple-choice format. We use multiple-choice scoring deliberately. It avoids LLM judges, which the Mirage of Hallucination Detection shows have weak agreement with each other, game label distributions, and are biased against short answers.
Mediator. The mediator $M$ is the residual-stream state over the core span at a band of middle layers. We choose the band on held-out, non-TruthfulQA true/false statements, as the layers where a linear probe best reads out statement truth. Choosing it off-benchmark keeps the "graph" from being fitted to the effect we want to measure. This matters because misspecified graphs are the known weakness of path-specific methods.
Estimands. With $Y(\cdot)$ computed by activation patching, we measure two quantities. The first is the direct (imitation) effect: the framed prompt run with the mediator patched from the neutral run, compared against the neutral run. The second is the mediated effect: the neutral prompt run with the mediator patched from the framed run, again compared against the neutral run.
$$ \text{DE} = Y(a, M(a')) - Y(a', M(a')), \qquad \text{IE} = Y(a', M(a)) - Y(a', M(a')). $$
An error is labelled knowledge-intact when three conditions hold. The total effect is large and negative. $\text{IE}$ is near zero, meaning the framed mediator still supports the truth. $\text{DE}$ carries most of the drop. An error is labelled knowledge-corrupted when $\text{IE}$ carries the drop, so that framing damaged the internal fact. An error is absent-knowledge when even the neutral run $Y(a', M(a'))$ prefers the falsehood.
Two-mediator extension. We add a second, "persona" mediator: states at the preamble and question-stem tokens. We apply the Harsanyi-dividend decomposition of Dividend Correction to split effects additively across the two mediators and the residual direct path. This tests whether "imitation" is carried by an identifiable stem representation.
Secondary mitigation. The output $Y(a', M(a))$ answers from the user's own framed-question fact state under a neutral output context. It is compared honestly against simply answering the paraphrase.
Baselines
Behavioral reference points. The "do nothing" point is the unmodified model on the original TruthfulQA questions. The naive behavioral competence test is the model's accuracy on our neutral rewrites $Y(a', M(a'))$. A behavioral evaluator could compute this without any causal machinery. If this test alone predicts which items are knowledge-intact as well as our estimands do, our diagnostic adds nothing.
Probe-disagreement diagnostic. This is the strongest competitor for the diagnostic claim. We train a linear truth probe on the same mediator layers, using the same held-out training statements and the same layer-selection budget. Items where the probe says "true" but the output says "false" are labelled knowing-but-imitating. It is the obvious correlational version of our question.
Representation-steering methods. For mitigation, the strongest existing method in the file is TruthFlow. It reports raising Llama-3-8B's TruthfulQA "True" score from 52.32% to 64.79%, with MMLU at 65.57% versus 65.77%. On Mistral-7B it reports 78.48% versus 75.31%. We rerun it on Llama-3-8B with the authors' settings, on the same TruthfulQA split. GRATH is a training-based reference: it reports Llama2-Chat-7B MC1 rising from 30.23% to 54.71% and MC2 from 45.32% to 69.10%. We include it on Llama2-Chat-7B.
Fairness of comparison. All methods get the same held-out statements for any fitting. None sees TruthfulQA labels during fitting. Tuning budgets are matched.
Ablations. Two ablations isolate our key ingredient, which is the normatively chosen mediator. The first patches a random mediator: same size, but from layers where the probe fails. The second patches the whole residual stream at the core span, which recovers "just answer the paraphrase."
What counts as a win. The diagnostic wins if its labels predict intervention success (defined below) better than probe disagreement. The mitigation wins only if it beats both the paraphrase baseline and TruthFlow's reproduced score at equal MMLU retention. If it does not, we report mitigation as a negative result and keep the diagnostic as the contribution.
Why not just…?
Why not just use a truth probe and look for probe/output disagreement? This is the real threat. A probe shows that truth information is linearly present, not that the model's computation uses it. A probe can also read framing cues correlated with truth. Our estimands are interventional: an item is knowledge-intact only if moving its framed-question state into a neutral run actually yields the true answer. We settle this empirically. On a held-out set, we test which labelling better predicts whether patching the framed mediator into a neutral output context yields the true answer. If probe disagreement predicts this as well, we concede that the causal machinery is not needed for diagnosis.
Why not just ask the neutral paraphrase and call that "knowing"? This is essentially the consistency approach of Agrawal et al.. It shows the model can answer under some phrasing. It cannot tell whether the framed question corrupted the model's fact state or only redirected the output, and that split is exactly what distinguishes the two explanations of inverse scaling. As a mitigation, paraphrasing is a strong baseline and may match $Y(a', M(a))$. That would happen whenever the framed question carries no useful extra information. We expect this outcome on most items and will report it rather than argue it away.
Why not just apply standard activation-patching mediation from interpretability? Mechanically, our estimator is activation patching. What the fairness literature adds is two things. The first is the discipline of fixing the allowed mediator in advance, off-benchmark, and treating that choice as normative, as Shpitser et al. argue. The second is the nested, additive path decomposition of Dividend Correction. The random-mediator and full-stream ablations show whether this discipline changes conclusions. If it does not, the contribution reduces to the empirical taxonomy.
Why not just steer toward truth with TruthFlow and skip diagnosis? Steering improves the score but says nothing about whether the model lacked the fact. That question is the one Type-I anthropocentrism raises about evaluation itself. Steering also produces no per-item competence label.
Proposed Experiments
Experiment 1: taxonomy of errors. We run the full taxonomy on Llama-3-8B, Mistral-7B and Llama2-Chat-7B, plus two sizes within one family, over all 817 TruthfulQA questions and the HotpotQA control pairs. The decisive check is whether knowledge-intact errors form a substantial share of TruthfulQA errors while being rare on controls. We have no prior number here, so the outcome is genuinely open. If the share is near zero, the Type-I worry does not apply to imitative falsehoods at this scale.
Experiment 2: inverse scaling. The decisive check is whether, across sizes, growth in errors comes from rising $\text{DE}$ with stable $\text{IE}$ (better imitation) or from rising $\text{IE}$ (corrupted knowledge).
Experiment 3: validity. The probe-versus-causal prediction test from the section above is decisive for whether the method earns its complexity. We add a robustness check in which the mediator band is shifted by plus or minus two layers. Labels that change under this shift would signal that the estimand is sensitive to graph misspecification.
Experiment 4: mitigation. We report TruthfulQA MC1/MC2 and HaluEval QA transfer for three outputs: $Y(a', M(a))$, the paraphrase baseline, and TruthFlow. Each is reported with MMLU retention alongside.
References
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- Anthropocentric bias and the possibility of artificial cognition
- Do Language Models Know When They're Hallucinating References?
- Path-Specific Counterfactual Fairness
- Avoiding Discrimination through Causal Reasoning
- Fair Inference on Outcomes
- Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs
- Dismantling Pathological Shortcuts: A Causal Framework for Faithful LVLM Decoding
- Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formats
- Treble Counterfactual VLMs: A Causal Approach to Hallucination
- Trustworthy Reasoning: Evaluating and Enhancing Factual Accuracy in LLM Intermediate Thought Processes
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
- Evaluating Evaluation Metrics – The Mirage of Hallucination Detection
- Path-Specific Counterfactual Fairness via Dividend Correction
- TruthFlow: Truthful LLM Generation via Representation Flow Correction
- GRATH: Gradual Self-Truthifying for Large Language Models