Editing Knowledge Representation of Language Model via Rephrased Prefix Prompts

Yuchen Cai, Ding Cao, Rongxi Guo, Yaqin Wen, Guiquan Liu, Enhong Chen, Jingyun Zhang

TL;DR

This paper introduces PSPEM, a novel method for editing knowledge within large language models (LLMs) using rephrased prefix prompts. PSPEM efficiently extracts and refines prompt information to guide LLM outputs, achieving near-perfect accuracy in knowledge editing and state-of-the-art results in attribute insertion, while maintaining fluency and consistency.

What Problem Does This Paper Address?

LLMs store vast amounts of factual knowledge within their parameters, but this knowledge can become outdated or contain biases. Traditional methods for updating this knowledge, like retraining or fine-tuning, are computationally expensive and inefficient. Existing knowledge editing techniques often struggle with generating appropriate text, and prompt engineering, while less resource-intensive, is opaque and requires significant manual effort to find effective prompts. This paper addresses the need for an efficient, generalizable, and less labor-intensive method for updating LLM knowledge that can also be used for reasoning.

Methods

The core of PSPEM lies in its three main components:

  • Prompt Encoder: This component extracts key information from the input prompt. It leverages the Transformer architecture of the LLM itself to obtain a compressed sentence representation from a specific layer's output.
  • Encoding Converter: This novel mechanism refines the compressed sentence representation from the Prompt Encoder. It uses multi-layer perceptrons (MLPs) to transform the representation into a more accurate sentence representation, essentially a set of word embeddings that focus on the crucial information within the prompt.
  • Alignment Techniques: PSPEM employs prompt alignment techniques to guide the LLM's generation process. This involves maximizing the probability of the target token and ensuring the generated text aligns with the original prompt's influence on continuation words, thereby guaranteeing fluency and consistency.

Crucially, PSPEM achieves these edits without altering the LLM's original weights, making it a "weight-preserved" method.

Datasets

The paper evaluates PSPEM on several established datasets:

  • Knowledge Editing:
    • ZsRE: A question-answering dataset used for evaluating knowledge editing.
  • COUNTERFACT: A benchmark curated from Wikipedia, designed to test LLMs' ability to reason with new knowledge and distinguish superficial changes from deeper ones.
  • Attribute Inserting:
    • BioBias: Contains biographies of non-celebrities with occupational themes, used to test the model's ability to infer occupations.
  • McRae: A dataset detailing concepts and their features, used to assess how well models can associate new attributes with existing concepts.

Key Insights and Ideas

  • Soft Prompts for Editing: PSPEM demonstrates that soft prompts, learned and refined through an encoding mechanism, can be a powerful tool for knowledge editing, offering a more nuanced approach than traditional hard prompts.
  • Lifetime Editing: The method is designed for "lifetime" use with just one training of the Encoding Converter, implying a persistent and adaptable editing capability.
  • Balancing Editing and Fluency: PSPEM successfully balances the need for accurate knowledge editing with the generation of fluent and consistent text, a common challenge in this field.
  • Mimicking Original Prompt Influence: The analysis shows that PSPEM's impact on the model's internal workings is highly similar to that of original prompts, suggesting it can effectively guide the model without introducing significant internal deviations.

Key Results

  • Knowledge Editing: PSPEM achieved nearly 100% editing accuracy on the COUNTERFACT dataset and demonstrated the highest level of fluency among the evaluated weight-preserved methods.
  • Attribute Inserting: PSPEM reached state-of-the-art performance on attribute insertion tasks, showing an ability to make reasonable inferences and generate text aligned with prompt information.
  • Comparison to Baselines: PSPEM consistently outperformed other weight-preserved methods like PREFIX PROMPT and REMEDI in various metrics, including accuracy, fluency, and consistency. It also showed competitive or superior performance compared to several weight-modified methods.

Limitations and Future Work

  • Hyperparameter Sensitivity: The paper notes that the best performance is achieved when the hyper-parameters $\lambda_1$ and $\lambda_2$ are set to 1, suggesting some sensitivity to these settings. $\lambda_1$ controls editing accuracy, while $\lambda_2$ influences text quality.
  • Further Exploration: The authors suggest that their work can serve as a catalyst for further research in developing more user-friendly and precise LM editing tools, implying that PSPEM is a foundational step.
  • Broader Applications: The paper anticipates that PSPEM's contributions will broaden the scope of prompt engineering in knowledge editing and LM applications across diverse domains.

Summary updated

Abstract

Neural language models (LMs) have been extensively trained on vast corpora to store factual knowledge about various aspects of the world described in texts. Current technologies typically employ knowledge editing methods or specific prompts to modify LM outputs. However, existing knowledge editing methods are costly and inefficient, struggling to produce appropriate text. Additionally, prompt engineering is opaque and requires significant effort to find suitable prompts. To address these issues, we introduce a new method called PSPEM (Prefix Soft-Prompt Editing Method), that can be used for a lifetime with just one training. It resolves the inefficiencies and generalizability issues in knowledge editing methods and overcomes the opacity of prompt engineering by automatically seeking optimal soft prompts. Specifically, PSPEM adopts a prompt encoder and an encoding converter to compress and refine key information in prompts and adopts prompt alignment techniques to guide model generation, ensuring text consistency and adherence to the intended structure and content. We have validated the effectiveness of PSPEM through knowledge editing and attribute inserting. On the COUNTERFACT dataset, PSPEM achieved nearly 100% editing accuracy and demonstrated the highest level of fluency. We further analyzed the similarities between PSPEM and original prompts and their impact on the model's internals. The results indicate that PSPEM can serve as an alternative to original prompts, supporting the model in effective editing.
BibTeX
@inproceedings{cai2024editing,
  author = {Yuchen Cai and Ding Cao and Rongxi Guo and Yaqin Wen and Guiquan Liu and Enhong Chen and Jingyun Zhang},
  title = {{Editing Knowledge Representation of Language Model via Rephrased Prefix Prompts}},
  booktitle = {ICIC (LNAI 4)},
  year = {2024},
  doi = {10.1007/978-981-97-5672-8\_39},
}
Figure