Is it Possible to Edit Large Language Models Robustly?

Xinbei Ma, Tianjie Ju, Jiyang Qiu, Zhuosheng Zhang, Hai Zhao, Lifeng Liu, Yulong Wang

Abstract

Large language models (LLMs) have played a pivotal role in building communicative AI to imitate human behaviors but face the challenge of efficient customization. To tackle this challenge, recent studies have delved into the realm of model editing, which manipulates specific memories of language models and changes the related language generation. However, the robustness of model editing remains an open question. This work seeks to understand the strengths and limitations of editing methods, thus facilitating robust, realistic applications of communicative AI. Concretely, we conduct extensive analysis to address three key research questions regarding consistency in realistic situations, the impact of prompt rephrasing, and the correlation between knowledge features and editing performance. Our experimental results uncover a substantial disparity between existing editing methods and the practical application of LLMs. On rephrased prompts that are complex and flexible but common in realistic applications, the performance of editing experiences a significant decline. Further analysis shows that more popular knowledge is memorized better, easier to recall, and more challenging to edit effectively.

Introduction

As Large Language Models (LLMs) increasingly serve as the foundation for "communicative AI"—including chatbots and autonomous agents—the ability to update their knowledge base without the prohibitive cost of full retraining has become essential. Model editing techniques, such as ROME and MEMIT, have emerged as promising solutions for surgically correcting errors or reflecting new information. However, current evaluation benchmarks often rely on simplistic, direct queries that do not reflect the complexity of real-world interactions. This study investigates the robustness of these edits when subjected to adversarial conditions, multi-turn dialogues, and rephrased prompts.

Overview of the model editing and evaluation framework, illustrating the transition from standard editing to adversarial testing.

Methodology and Evaluation Framework

The researchers evaluated several state-of-the-art editing methods, including ROME, MEMIT, MEND, SERAC, and IKE, primarily utilizing the Llama-2-7B-chat model. The evaluation framework was expanded beyond standard benchmarks (CounterFact, zsRE, MQUAKE-T) to include three primary "attacking" strategies:

  • Dialogue Simulation: A GPT-4 "user" engaged the edited model in multi-turn conversations to test for consistency.
  • Attacking Prompts: Models were challenged with contextual noise (Wikipedia snippets), query rephrasing (e.g., cloze formats), and explicit "Raising Doubts" prompts (e.g., "Are you sure?").
  • Popularity Analysis: The study quantified "knowledge popularity" using Wikipedia page views, knowledge graph connectivity, and co-occurrence frequencies to determine if certain facts are inherently more resistant to change.

The Reversion Phenomenon and Hallucination

A primary finding of this research is the "Reversion Phenomenon." Even when an edit appears successful on a direct prompt, the model frequently reverts to its original, pre-edited parametric memory during a conversation. In dialogue simulations, 38% of samples reverted to the old, incorrect answer.

An example of knowledge reversion and hallucination during a multi-turn dialogue simulation.

Furthermore, when the model encounters a conflict between its edited memory and its original training data, it often defaults to hallucination. 78% of dialogue samples resulted in the generation of unreal or nonsensical content. This suggests that the model's internal "prior" remains a powerful force that surgical edits fail to fully suppress.

The Popularity Paradox

The study highlights a significant correlation between the popularity of a fact and the difficulty of editing it. Knowledge that is highly connected or frequently mentioned in the pre-training corpus is memorized more deeply. Consequently, these "popular" facts are significantly harder to edit robustly. While a parameter tweak might change the model's output for a specific query, the underlying original knowledge remains accessible and often resurfaces when the model is pushed or the query is rephrased.

Fragility Under Attack

The performance of "locate-then-edit" methods like ROME and MEMIT proved highly sensitive to prompt phrasing. While these methods achieved nearly 100% accuracy on direct prompts in the CounterFact dataset, their performance dropped to 20-50% when faced with rephrased queries or added context. The "Doubt" test was particularly effective; simply questioning the model's new knowledge caused many models to immediately apologize and revert to the original fact.

Conclusion

The results demonstrate a massive gap between "edit success" on isolated benchmarks and "communicative robustness" in realistic applications. For model editing to be viable in production environments, new methods must be developed that can maintain consistency across diverse conversational contexts and effectively override strong pre-trained priors. Future research should focus on robustness augmentation and ensuring that reversion does not compromise safety or privacy.

Summary updated

Abstract

Large language models (LLMs) have played a pivotal role in building communicative AI to imitate human behaviors but face the challenge of efficient customization. To tackle this challenge, recent studies have delved into the realm of model editing, which manipulates specific memories of language models and changes the related language generation. However, the robustness of model editing remains an open question. This work seeks to understand the strengths and limitations of editing methods, thus facilitating robust, realistic applications of communicative AI. Concretely, we conduct extensive analysis to address the three key research questions. Q1: Can edited LLMs behave consistently resembling communicative AI in realistic situations? Q2: To what extent does the rephrasing of prompts lead LLMs to deviate from the edited knowledge memory? Q3: Which knowledge features are correlated with the performance and robustness of editing? Our experimental results uncover a substantial disparity between existing editing methods and the practical application of LLMs. On rephrased prompts that are complex and flexible but common in realistic applications, the performance of editing experiences a significant decline. Further analysis shows that more popular knowledge is memorized better, easier to recall, and more challenging to edit effectively.
BibTeX
@inproceedings{ma2024is,
  author = {Xinbei Ma and Tianjie Ju and Jiyang Qiu and Zhuosheng Zhang and Hai Zhao and Lifeng Liu and Yulong Wang},
  title = {{Is it Possible to Edit Large Language Models Robustly?}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.906},
}
Figure