Elastic Weight Removal for Faithful and Abstractive Dialogue Generation

Nico Daheim, Nouha Dziri, Mrinmaya Sachan, Iryna Gurevych, Edoardo M. Ponti

Introduction

Large Language Models (LLMs) frequently generate "hallucinations"—responses that are factually incorrect or ungrounded in the provided source knowledge. While techniques like task arithmetic attempt to "unlearn" these behaviors by subtracting weights of models trained on negative examples, they often treat all parameters as equally important. This blunt approach can lead to "catastrophic interference," where the model's general conversational capabilities are degraded. Furthermore, simply discouraging hallucinations often forces models to become overly extractive, merely copying source text rather than generating natural, abstractive responses.

Methodology: Elastic Weight Removal (EWR)

We introduce Elastic Weight Removal (EWR), a method for surgical parameter modification. EWR identifies and subtracts specific weights responsible for undesirable behaviors, weighted by their Fisher Information Matrix (FIM). The FIM serves as a proxy for parameter importance, allowing for a more precise removal of "bad" behaviors compared to standard task arithmetic.

The process involves:

  1. The Anti-Expert: Training a "negative expert" on a dataset of hallucinations.
  2. Importance Weighting: Using the diagonal of the FIM to determine which parameters are most critical for the hallucination behavior.
  3. Parameter Interpolation: Merging the base model with the anti-expert task vector, moving parameters that contribute heavily to hallucinations more aggressively while preserving those essential for general language modeling.
  4. Multi-Objective Steering: Simultaneously subtracting a hallucination anti-expert and adding an abstraction expert to balance the trade-off between factuality and naturalness.
Trade-off analysis between faithfulness (Critic), abstractiveness (Density), and performance (BLEU).

Experimental Results

EWR was evaluated across several benchmarks, including Wizard-of-Wikipedia (WoW) and DSTC9/11. The results indicate that EWR significantly improves faithfulness (measured by the "Critic" score) without the inference-time overhead of traditional ensemble methods like DExperts.

Main results comparing EWR against baselines on WoW and DSTC9 datasets.

As shown in the results table, EWR combined with control tokens (CTRL) achieves state-of-the-art performance. Human validation further supports these findings: EWR-steered models were rated as "Attributable" (faithful) 96.8% of the time on WoW, a substantial improvement over the 72.3% achieved by the vanilla Flan-T5 base. The method also proved robust on "unseen" topics, suggesting that the identified hallucination parameters are somewhat universal across domains.

Discussion and Future Work

A key insight from our analysis is the inherent trade-off between faithfulness and extractiveness. EWR is unique in its ability to tune these dimensions independently by combining multiple task vectors. Because the modifications occur at the weight level, the final model maintains the original architecture's inference speed, avoiding the computational costs associated with multi-model ensembles.

Future work will explore the application of EWR to other undesirable traits, such as toxicity and bias, and investigate more sophisticated metrics for defining abstractiveness beyond simple n-gram overlap.

Summary updated

BibTeX
@inproceedings{daheim2024elastic,
  author = {Nico Daheim and Nouha Dziri and Mrinmaya Sachan and Iryna Gurevych and Edoardo M. Ponti},
  title = {{Elastic Weight Removal for Faithful and Abstractive Dialogue Generation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.393},
}
Figure