Subtractive Repair: Can Removing Computation Alone Fix an LLM's Factual Errors?

Some wrong answers from a language model are not caused by missing knowledge. The model stores the right fact, but another internal pathway overrides it. A familiar example is a high-frequency association drowning out a rarer one. This note proposes using a constraint borrowed from lottery-ticket research as a measuring instrument. The idea is to learn a binary mask, a per-unit on/off switch over a frozen pretrained model's attention heads and MLP neurons, that may only switch units off. The mask is trained to make the model answer factual questions correctly while leaving its behavior on ordinary text unchanged.

Because a mask cannot write new information into the weights, any factual error it fixes on held-out questions must have been a suppression error: the correct answer was already computable and was being blocked. That turns a vague question, whether hallucination is a knowledge gap or a routing failure, into one that can be measured. It also gives a repair that is tiny (one bit per unit), reversible, and unable by construction to memorize answers.

Core Claim

We claim that a measurable share of closed-book factual errors in 7B-scale models are suppression errors. We further claim that these errors can be fixed by a single shared removal mask covering roughly 1–2% of structural units, with negligible damage to perplexity and to context-retrieval ability.

The mask's gains should be concentrated on one class of errors, which we call latent-correct errors. These are questions where the greedy answer is wrong but the correct answer appears among the model's own samples or carries nontrivial probability. On latent-absent errors, where the model never produces the right answer, the mask should give near-zero gain. If that split appears, subtractive repair is a tool for diagnosing why a model hallucinates, not only for reducing hallucination.

Relation to Prior Work

The mechanism comes from untrained-subnetwork research. What's Hidden in a Randomly Weighted Neural Network? introduced edge-popup. Edge-popup keeps a real-valued score per frozen weight, thresholds the scores into a hard mask in the forward pass, and passes gradients straight through the threshold, a trick called a straight-through estimator. Deconstructing Lottery Tickets showed that masks alone act as a training signal. The Strong Lottery Ticket Hypothesis for Multi-Head Attention Mechanisms extended mask search to attention and GPT-2 language modeling. One Network, Many Masks and AdvMask showed that masks over frozen pretrained weights can adapt behavior without forgetting.

We reuse their optimizer but change the setting in three ways. We start from a pretrained LLM, not random weights, because a random network contains no facts to unblock. We restrict the mask to removal from an all-on start. And we treat the result as evidence about where errors come from, not only as a compression method.

Four papers motivate the hypothesis. Pierce the Mists, Greet the Sky found that dominant associations suppress subordinate facts and that ablating attributed edges flips answers back to the truth. That work used per-fact circuit attribution on Pythia-410m and GPT-2 with synthetic knowledge. We ask whether one learned mask generalizes across many real facts at 7B scale. Lighter, yet More Faithful found that generic pruning (magnitude, SparseGPT, Wanda) reduces summarization hallucination. It did so by weakening parametric knowledge, which is the opposite of what we want in closed-book QA, where knowledge must be kept and only the overriding pathway removed. PIP-KAG scaled down whole mid-to-deep FFN layers to resolve conflicts between context and memory, then fine-tuned. Our mask is unit-level, needs no weight update, and targets memory-only errors.

One paper constrains the design. Retrieval Head Mechanistically Explains Long-Context Factuality showed that removing about 20 specific heads drops Needle-in-a-Haystack accuracy from ~95% to 63.6%. A removal mask must therefore be explicitly checked against this kind of collateral damage.

The closest competitor in outcome is CASAL. CASAL trains one sub-module's continuous weights toward abstention and reports 30–40% hallucination reduction on TriviaQA, PopQA and EntityQA. CASAL reduces hallucination mainly by having the model say "I don't know." Our method tries to recover the correct answer, and it changes no weights.

Research Program

The trainable object is a score $s_u$ for every attention head and every MLP neuron $u$ in a frozen model. The mask is $m_u = 1[s_u > \tau]$, with scores initialized above threshold so that the starting model is the original. We use structured units rather than individual weights. This keeps search memory small, which addresses the memory cost noted for edge-popup, and it makes the found units easy to interpret. The objective is

$$\mathcal{L}(s) = \mathbb{E}_{(q,a)\sim D_{\text{fix}}}\big[-\log p_m(a\mid q)\big] + \beta\, \mathbb{E}_{x\sim D_{\text{gen}}}\,\mathrm{KL}\big(p_\theta(\cdot\mid x)\,\|\,p_m(\cdot\mid x)\big) + \gamma \sum_u \sigma(-s_u).$$

The terms are as follows. $p_\theta$ is the original model and $p_m$ the masked model. $D_{\text{fix}}$ contains questions with gold answers $a$. $D_{\text{gen}}$ is general text plus long-context retrieval prompts. $\sigma(-s_u)$ is a smooth count of removed units. $\beta$ and $\gamma$ trade off preservation against sparsity.

$D_{\text{fix}}$ deliberately mixes three kinds of questions: ones the model already answers correctly, latent-correct errors, and latent-absent errors. This stops the mask from learning a shortcut. Abstention is never rewarded.

Questions are labeled by drawing 20 samples per question from the original model. A question is latent-correct if greedy decoding is wrong but at least one sample is right. It is latent-absent if no sample is right.

Train and test are split by entity, so that a test question never shares a subject with a training question. Any held-out gain therefore reflects a general change in routing, not memorized answers.

We will run on LLaMA-2-7B and Mistral-7B, both studied in Lighter, yet More Faithful. We will also run Pythia-410m, where the overshadowing-circuit results of Pierce the Mists let us check whether the learned mask removes the same heads that attribution flags.

Baselines

The naive reference is the unmodified model with greedy decoding. All other methods are reported as changes from it.

The strongest existing method is CASAL. The file gives it a 30–40% hallucination reduction on TriviaQA, PopQA and EntityQA (CASAL). We will rerun it ourselves on the same models with the same entity-held-out splits and the same number of training examples, matching the 640-example regime it reports. CASAL's reduction comes partly from abstention, so we will report it on two separate axes: accuracy among attempted answers and the abstention rate.

Beating CASAL means one of two outcomes. The mask could match its hallucination reduction while increasing correct answers rather than abstentions. Alternatively, it could exceed CASAL on correct answers to latent-correct questions at equal perplexity.

The strongest weight-updating competitor is LoRA fine-tuning on the same $D_{\text{fix}}$, with the same KL preservation term, the same optimizer steps and a matched hyperparameter search budget. The file reports no LoRA result on these QA sets beyond CASAL's statement that LoRA needs 20x more data, so we will produce one.

Generic pruning is represented by Wanda and SparseGPT at the same removed-unit budget. This tests whether any sparsity helps. No closed-book QA result for these methods was found on Lacuna; the summarization result is HRR < 1.0 in 284 of 300 comparisons.

PIP-KAG-style layer scaling, which multiplies FFN layers 20–29 by $\lambda<1$ with no fine-tuning, tests whether coarse muting is enough.

A circuit-attribution baseline removes the top-$k$ units ranked by attribution scores of the kind used in Pierce the Mists, with $k$ matched to the mask size.

The key ablation is a random mask with the same per-layer counts. It shows how much of the gain comes from which units are removed. A second ablation drops the KL term, to show what preservation costs.

Why not just…?

Why not just fine-tune with LoRA on the same questions? This is the real threat. On in-distribution questions LoRA will likely win, because it can store answers. The entity-held-out split is what settles the comparison: LoRA's ability to add knowledge does not help on unseen entities, and the mask cannot add knowledge at all. If LoRA matches the mask on held-out latent-correct questions and matches it on WikiText-103 perplexity and Needle-in-a-Haystack, the repair claim reduces to "a cheaper adapter." The diagnostic claim, that gains are confined to latent-correct errors, would still be testable by applying the same split analysis to LoRA.

Why not just prune with Wanda or SparseGPT? Lighter, yet More Faithful shows that these methods reduce hallucination by weakening parametric memory. In a closed-book setting with no source text, that should lower accuracy rather than raise it, and the paper reports higher perplexity. The matched-budget comparison tests this directly.

Why not just ablate the heads that circuit attribution flags? Attribution is cheaper and may find the same units. Pierce the Mists did this per fact on small models. If top-$k$ attribution removal matches the learned mask on held-out entities, we will report that the optimizer is unnecessary, and the scientific claim would still stand. Measuring the overlap between the learned mask and the attributed units on Pythia-410m makes this comparison precise.

Why not just sample more or self-check at inference? Self-checking is unreliable: ChatGPT reaches only 58.53% self-detection on HaluEval summarization (HaluEval), and indirect queries multiply cost (Do Language Models Know When They're Hallucinating References?). Majority voting over samples is a fair reference point for latent-correct questions in particular, so we include self-consistency with 20 samples as a cost-unmatched upper reference. A single-pass mask that approaches it would be a useful result.

Proposed Experiments and Decisive Checks

Evaluation uses TriviaQA, PopQA and EntityQA for closed-book accuracy, abstention and hallucination rate. TruthfulQA is used for imitative falsehoods, where the best GPT-3 result with a helpful prompt is 58% truthful (TruthfulQA); that adversarial setting is a natural place to look for override errors. WikiText-103 perplexity serves as the fluency check. Needle-in-a-Haystack, with an unmasked reference of ~95%, and ExtractQA are the retrieval-damage checks.

Three checks decide the claim.

The first is the gain split. On held-out entities, accuracy gains should fall predominantly on latent-correct questions, with latent-absent gains statistically indistinguishable from the random-mask ablation. Gains spread evenly across both classes would suggest the mask is doing something other than unblocking, such as changing answer format.

The second is collateral damage. Perplexity must rise by less than the increase seen with Wanda at the same budget, and Needle-in-a-Haystack accuracy must stay near the unmasked ~95% rather than falling toward the 63.6% caused by retrieval-head removal. The removed set should also be checked for overlap with known retrieval heads.

The third is the frequency link. On PopQA, which is stratified by entity popularity, the mask should help most on low-popularity entities that share a subject with a high-frequency association. This is the overshadowing pattern, and finding it would connect the learned mask to the mechanism described in Pierce the Mists.

If all three checks hold, a removal mask both repairs the model and measures how much hallucination is caused by suppression. If the first check fails, we learn that closed-book errors at this scale are mostly genuine knowledge gaps, which is a useful negative result for the pruning-for-faithfulness line of work.