Large language models often know a fact when asked one way and invent an answer when asked another way. "Where did Einstein die?" gets the right city, while "What was the site of Einstein's death?" can get a confident wrong one. This note proposes to find the layer where a model's internal representation of the question stops depending on how the question was phrased. We then train a small low-rank adapter (LoRA, a cheap add-on that learns a low-rank correction to frozen weights) in the layers just before that point. Its only job is to make a structurally rephrased question produce the same internal state there that the model already produces for the phrasing it handles well. The adapter teaches no new facts. It uses no gold answers, only the model's own states under its preferred phrasing. Because it never touches the layers that store facts, it should avoid the usual trade-off in which fixing hallucinations damages correct knowledge.
The opportunity matters because rephrasing sensitivity is common, measurable, and different from not knowing a fact. A model that answers correctly under one phrasing has shown that the fact is stored. When it fails under another phrasing, the failure lies in how the question is read, not in what the model knows. That is a narrower and more fixable target than "hallucination" in general. It also ties together two findings that have not been connected: models lock onto rigid syntactic templates early in pre-training, and factual recall is brittle under paraphrase.
Core Claim
We claim that for closed-book factual questions, most failures under rephrasing are failures of query formation. These failures can be found by causal patching and repaired by a representation-alignment adapter placed only in the layers where they occur. If the claim holds, the adapter should do three things. It should close most of the accuracy gap between canonical and rephrased forms on facts the model already knows. It should generalize to syntactic operations it never saw in training, such as clefting or nominalization when trained only on passivization and fronting. It should leave canonical-form accuracy, calibration, and output fluency unchanged. The claim is falsifiable. If patching the canonical-form state into the rephrased run does not restore the right answer at any early or middle layer, the failure lies in late-layer knowledge competition, and the method should not work.
What Prior Work Establishes and Leaves Open
Two papers motivate the problem. Test-time Augmentation for Factual Probing showed behaviorally that rephrasing flips factual recall. It reduced the effect by averaging answers over up to 30 paraphrases per query, which multiplies inference cost about 30-fold, and its rule-based paraphrases hurt the largest model it tested. Improving the Robustness of Question Answering Systems to Question Paraphrasing showed that syntactic reordering drops BERT question-answering F1 from 89.31 to 63.18 on adversarial paraphrases. Retraining all parameters on 25,000 paraphrases recovered it only to 73.85. Neither paper looks inside the model, and both treat the fix as more data or more compute.
On the syntax side, Detection and Measurement of Syntactic Templates in Generated Text found that models acquire part-of-speech templates within the first 4 billion pre-training tokens, and that 76% of templates in model text are inherited from pre-training, against 35% for human text. This suggests models are tuned to a narrow set of syntactic frames, which could explain why unusual phrasings disrupt recall. That paper is observational and does not connect templates to factual circuits. Towards Better Characterization of Paraphrases contributes two pieces we reuse. Word Position Deviation measures how far shared words move between two sentences and so quantifies structural change. The same paper also warns that 77% of MRPC paraphrase pairs contain entity contradictions, so paraphrase data must be checked for changed facts.
On the interpretability side, ReDeEP found that in retrieval-augmented generation, late feed-forward layers "over-add" stored knowledge and override correctly attended context. It fixes this at inference by boosting attention heads and damping those layers. Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation places relevance decisions early and knowledge contestation late. Both papers constrain our design. They suggest that the late layers are where facts compete, so an intervention that avoids those layers should be less likely to corrupt knowledge. They also raise the alternative hypothesis that our patching experiment must rule out. SHINE perturbs entity embeddings with Gaussian noise to tell known facts from fabricated ones. We borrow its distinction between "the model knows but misfires" and "the model does not know", and we replace noise with real syntactic perturbation. CLAP shows that hallucination signals spread across layers, which supports reading states at several depths.
On the adaptation side, SRM-LoRA reshapes LoRA gradients in feed-forward layers away from "hallucination-sensitive" coordinates. It is trained on HaluEval contrastive pairs and does not test syntactic variation. On the Calibration of Large Language Models and Alignment found that LoRA preserves calibration better than full fine-tuning. That result supports using LoRA and gives us a calibration check. Our proposal differs from SRM-LoRA in three ways: what is targeted (reading the question rather than generic hallucination), where the adapter sits (attention and residual layers before fact lookup rather than the feed-forward stores), and what supervision is used (the model's own canonical states rather than labeled hallucinations).
Research Program
The first component is a structural rephrasing set. We start from the Wikidata relational facts of Test-time Augmentation for Factual Probing (12,500 facts across 25 relations) and from PopQA from When Not to Trust Language Models. For each question we generate rephrasings using named syntactic operations: passivization, fronting, clefting, nominalization, and embedding inside a subordinate clause. We keep a rephrasing only if three conditions hold. Its Word Position Deviation from the canonical form must exceed a threshold, which rules out mere synonym swaps. The subject and object entities must be string-identical. An entailment check in both directions must pass. We also sample a few hundred rephrasings for human checking. A fact counts as known if the model answers correctly under the canonical template. The quantity of interest is the rephrasing gap: the share of known facts that are answered wrongly under a rephrasing.
The second component is localization. For each known-but-flipped fact, we run the rephrased prompt and replace the hidden state at the final token and at the last subject token, at layer $l$, with the corresponding state from the canonical run. We then record whether the correct answer is restored. This yields a recovery curve over depth. Our hypothesis predicts that recovery is high from some early-to-middle layer $l^*$ onward. The competing hypothesis, adapted from ReDeEP, predicts recovery only when late feed-forward outputs are patched. We will also report how the intrinsic dimensionality profile described in Probing the Internal Representations of Large Language Models shifts between forms. This is descriptive context only.
The third component is the adapter. We place LoRA only on attention and output projections in layers below $l^*$. We train it on known facts with
$$\mathcal{L} = \sum_{l \in S} \big\| h_l(p) - \mathrm{sg}[h_l(c)] \big\|^2 \; - \; \mu \sum_{l \in S} \big\| h_l(p) - \mathrm{sg}[h_l(c')] \big\|^2 ,$$
where $h_l(\cdot)$ is the residual state at the final token at layer $l$, and $p$ is a rephrasing of the canonical question $c$. The second term uses $c'$, a canonical question about the same subject but a different relation, as a negative example. This keeps the adapter from collapsing distinct questions onto one state. $S$ is the set of layers selected by localization, $\mathrm{sg}$ means the target is held fixed, and $\mu$ weights the negative term. In practice the negative term is bounded, for example by a margin, so that it cannot grow without limit. The canonical run uses the frozen model, so the target never drifts. No answer labels are used.
The fourth component turns the analysis around. We measure whether existing anti-hallucination adapters change the rephrasing gap and whether they make output syntax more rigid. The adapters are SRM-LoRA, contrastive LoRA, and plain LoRA trained on HaluEval-QA. Output rigidity is measured with the template rate from Detection and Measurement of Syntactic Templates in Generated Text. This directly tests the untested concern that hallucination fixes and syntactic rigidity are entangled.
Baselines
The naive reference point is the unmodified model, Llama-3.1-8B-Instruct, chosen because SRM-LoRA reports on it, with LLaMA-2-7B as a second model. It is evaluated on the same canonical and rephrased prompts. It defines the rephrasing gap that every method must shrink. Beating it means a statistically significant reduction in the gap on held-out facts, with canonical accuracy within one point of the base model.
The first strong competitor is paraphrase data augmentation in the spirit of Improving the Robustness of Question Answering Systems to Question Paraphrasing. Here a standard LoRA (rank and placement as in common practice, attention and feed-forward) is trained with next-token loss on rephrased questions paired with the model's own canonical answers. That paper's full-retraining version raised adversarial BERT F1 from 63.18 to 73.85. This is the most natural thing a practitioner would try, so it is the main competitor on matched data.
The strongest competitor on this exact question is output-level consistency training. It uses the same LoRA rank and the same training pairs, but the loss is the KL divergence between the next-token distributions under $p$ and under the frozen model on $c$, with the adapter at default placement. This differs from our method in two respects at once: where the adapter sits and where the loss is applied. We also include the inference-time option, test-time augmentation over 30 rephrasings from Test-time Augmentation for Factual Probing. For that method we found no reported absolute accuracy numbers on Lacuna for its Wikidata corpus. We will therefore rerun it and report it at its own roughly 30-fold inference cost. For general hallucination we compare with SRM-LoRA, which reports a 10.04% HaluEval hallucination rate on Llama-3.1-8B-Instruct against 11.46% for plain LoRA. Our adapter should not raise that rate.
Two ablations isolate the key ingredient. The first keeps our representation loss but moves the adapter into the late feed-forward layers. The second keeps our placement but swaps in the output KL loss. If localized placement matters, the first ablation should hurt canonical accuracy or calibration. If representation-level targets matter, the second should generalize worse to held-out syntactic operations. A third control places the adapter at random layers with the same parameter count. For fairness, all trained methods use identical training pairs, rank, trainable-parameter count, optimizer steps, and an identical hyperparameter search budget of the same number of configurations, selected on the same validation split of held-out facts.
Why not just…?
Why not just train on paraphrases with an ordinary LoRA? This may close the gap for the syntactic operations seen in training. The question is whether it transfers. The augmentation result in Improving the Robustness of Question Answering Systems to Question Paraphrasing recovered only part of the lost F1 (63.18 to 73.85, against 89.31 on original questions). Training on answer tokens also lets the adapter memorize facts rather than learn to read questions. The held-out-operation and held-out-fact splits test this directly. If augmentation matches us on both splits, we will report that the localization machinery is unnecessary.
Why not just average over many paraphrases at test time? It works for smaller models, but at about 30 times the inference cost. Test-time Augmentation for Factual Probing also reported a dip on its largest model, caused by noisy paraphrases. It also requires generating paraphrases at deployment. We compare at matched inference cost and at its own cost, and report both.
Why not just add an output-consistency loss and skip the interpretability? This is the real threat, because it targets the same invariance with less machinery. Our bet is that matching output distributions lets the adapter change any layer, including the feed-forward layers that store facts, and so invites the collateral damage SRM-LoRA was built to avoid. We also bet that intermediate-state targets carry more signal per example than a single next-token distribution. The second ablation settles this. If output KL at default placement matches us on held-out operations, canonical accuracy, HaluEval, and expected calibration error, then the contribution reduces to the diagnostic finding and we will say so.
Why not just use SRM-LoRA or ReDeEP-style steering? Both target a different failure. SRM-LoRA protects against generic hallucination sensitivity learned from HaluEval pairs. ReDeEP addresses late-layer overriding of retrieved context. Neither sees syntactic variation. Component four measures whether they change the rephrasing gap at all.
Proposed Experiments and Decisive Checks
The first decisive check is the patching curve on known-but-flipped facts in the Wikidata and PopQA sets. If replacing states before layer $l^*$ restores the correct answer in most flipped cases, the query-formation hypothesis stands. If restoration requires patching late feed-forward outputs, it falls, and we would pivot to a late-layer analysis aligned with ReDeEP.
The second check is held-out generalization, reported as the rephrasing gap. Splits are by syntactic operation, by relation, and by fact. We also report gap reduction as a function of Word Position Deviation, so that structural rephrasings are shown separately from lexical ones.
The third check is collateral damage. It covers canonical accuracy, HaluEval-QA hallucination rate against the SRM-LoRA figures above, expected calibration error on MMLU following On the Calibration of Large Language Models and Alignment, and the output template rate on open generation.
The fourth check transfers to context-based question answering on the 1,062 human-verified paraphrases and the 56 adversarial questions of Improving the Robustness of Question Answering Systems to Question Paraphrasing, adapted as prompts for a generative model. Those are sized for exploration, not strong statistical claims. Last, we run the four-way comparison of base, plain LoRA, contrastive LoRA, and SRM-LoRA on the rephrasing gap and the output template rate. This tells us whether current hallucination fixes make models more syntactically rigid.
References
- Test-time Augmentation for Factual Probing
- Improving the Robustness of Question Answering Systems to Question Paraphrasing
- Detection and Measurement of Syntactic Templates in Generated Text
- Towards Better Characterization of Paraphrases
- ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability
- Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation
- Probing LLM Hallucination from Within: Perturbation-Driven Approach via Internal Knowledge
- Cross-Layer Attention Probing for Fine-Grained Hallucination Detection
- SRM-LoRA: Sub-Riemannian-Style Updates for Mitigating LLM Hallucination in Low-Rank Adaptation
- On the Calibration of Large Language Models and Alignment
- When Not to Trust Language Models: Investigating Effectiveness and Limitations of Parametric and Non-Parametric Memories
- Geometry of Decision Making in Language Models