Large Language Models (LLMs) are impressive conversationalists, but they have a persistent habit of "hallucinating"—generating information that sounds confident but isn't supported by the source documents they are supposed to be grounded in. While we can try to train them to be more faithful, this often results in models that simply copy-paste text verbatim, losing the "abstractive" (paraphrasing) quality that makes them useful. This research introduces Elastic Weight Removal (EWR), a novel way to "edit" a model’s knowledge by surgically removing the parameters responsible for hallucinations.
The Problem: Faithfulness vs. Abstraction
In document-grounded dialogue, a model is given a snippet of knowledge (e.g., a Wikipedia entry) and asked to answer a user query based only on that snippet. However, models often drift into "unfaithful" territory, adding outside info or contradicting the source. Existing solutions usually fall into two camps:
- Training-time fixes: Using special "control tokens" or reinforcement learning to discourage lying. These are often computationally expensive.
- Model Merging (Task Arithmetic): Subtracting a "hallucination task vector" from the model. The problem here is that it treats all parameters as equally important, which can lead to "catastrophic interference"—breaking the model's ability to speak fluently while trying to fix its honesty.
Methods: Surgical Precision with EWR
The core idea of EWR is inspired by "anti-experts." The process involves:
- The Anti-Expert: Training a version of the model specifically to hallucinate using datasets of known errors.
- The Task Vector: Calculating the difference between the base model and this "liar" model.
- Fisher Information Matrix (FIM): Instead of subtracting the task vector uniformly, EWR uses the FIM to estimate the importance of each parameter. If a specific weight is highly influential in producing hallucinated text, EWR gives it a higher scaling factor for removal.
- The Expert: An "abstraction expert"—a model trained on high-quality paraphrases—can also be added to ensure the model doesn't just become a copy-paste machine.
This allows for a "weighted subtraction" that removes the bad behavior with surgical precision while leaving the model's general language capabilities intact. As shown in Figure 3, researchers can navigate the trade-off between faithfulness and abstraction by adjusting the scaling factors for these experts.
Experimental Results
The researchers evaluated EWR across several challenging benchmarks, including Wizard-of-Wikipedia (WoW) and DSTC9, using metrics such as BLEU for performance and a "Critic" score to measure hallucination (where lower is better).
Key Findings
- Superior Faithfulness: On the WoW dataset, EWR significantly outperformed standard Flan-T5 and other baselines like DExperts and Quark. It reduced the Critic score by over 6 points compared to the base model.
- State-of-the-Art (SOTA) when Combined: When EWR was applied on top of other methods like CTRL (control tokens), it achieved the best results across the board, reaching a Critic score as low as 1.2 on DSTC9 (see Figure 6).
- Weight Importance Matters: Uniformly subtracting weights (Task Arithmetic) is a blunt instrument. EWR proves that by identifying which weights are "guilty" of hallucination via the FIM, you can improve faithfulness without destroying fluency (BLEU scores).
- Human Validation: In a study with expert annotators, EWR-modified models were rated as significantly more "attributable" (faithful to the source) than vanilla models and even outperformed previous SOTA methods by a statistically significant margin.
Conclusion and Future Work
EWR acts as a form of "unlearning," effectively removing the influence of bad data from the final model's weights without retraining from scratch. While the experiments were conducted on models up to 1 billion parameters (Flan-T5 Large), the computational cost of calculating the FIM for 70B+ parameter models remains a challenge for future scaling. Additionally, the quality of the "anti-expert" depends on the quality of the hallucinated data used to train it, suggesting that more sophisticated synthetic data generation could further refine the method.
Summary updated