InstructEd: Soft-Instruction Tuning for Model Editing with Hops

Xiaoqi Han, Ru Li, Xiaoli Li, Jiye Liang, Zifang Zhang, Jeff Z. PanACL 2024 · DOI

Large Language Models (LLMs) frequently require updates to correct outdated or erroneous information. Traditional model editing (ME) techniques often struggle with "portability"—the ability to apply an edited fact to related multi-hop reasoning tasks. This paper introduces InstructEd, a novel soft-instruction tuning approach that shifts the focus from modifying knowledge storage (FFNs) to optimizing knowledge interaction (Attention). By inserting learnable soft-instruction prefixes and employing a relevance-based re-weighting scheme, InstructEd achieves state-of-the-art performance in complex reasoning tasks while remaining highly parameter-efficient.

The Portability Gap in Model Editing

Model editing aims to rectify specific "bugs" in LLMs without the prohibitive cost of full retraining. Current methodologies generally fall into two categories, both of which face significant hurdles in generalizing new knowledge:

  1. Modification-based methods: These techniques typically alter model weights within Feed-Forward Networks (FFNs). While effective at memorizing a specific fact (e.g., "Messi plays for Inter Miami"), they often fail to generalize to related queries, such as "What city does Messi's team play in?"
  2. Memory-based methods: These store facts in external databases. However, they frequently suffer from "knowledge conflicts," where the model's internal pre-trained knowledge overrides the retrieved fact, or "error propagation" during multi-hop reasoning chains.
Current model editing methods often ignore whether a question is relevant to the edited fact, leading to 'Excessive Knowledge' issues.

The core issue is the portability gap: the inability of edited models to utilize new information in one-hop or multi-hop scenarios.

InstructEd: Shifting Focus to Interaction

InstructEd proposes a paradigm shift: treating portability as an interaction problem rather than a storage problem. Instead of tweaking the FFNs (the "knowledge base"), InstructEd modifies the Attention module (the "interaction engine").

Soft-Instruction Prefixes

The method introduces learnable vectors, termed "soft instructions," as prefixes to input tokens within the attention layers. These prefixes act as a bridge, guiding the model on how to integrate the new fact with the user's query.

Strategic Layer Targeting

Through ablation studies, the authors found that the "middle" layers (specifically layers 10-20 in a 32-layer model) are the most critical for semantic interaction. Editing early layers fails to capture sufficient semantics, while editing deep layers can negatively impact the model's general capabilities.

Relevance-Based Re-weighting

To maintain "locality"—ensuring the model doesn't apply edits to irrelevant queries—InstructEd employs a re-weighting mechanism. It calculates a relevance score based on the cosine similarity between the query and the instruction, ensuring the model only "listens" to the edit when appropriate.

The InstructEd architecture (InsEditor) uses soft-instruction prefixes within the attention layers of a frozen LLM to facilitate better knowledge interaction.

Experimental Evaluation

The researchers evaluated InstructEd across several benchmarks:

  • CounterFact (CT): Testing the model's ability to prioritize new, counter-intuitive facts over pre-trained knowledge.
  • ZsRE: A standard benchmark for one-hop relation editing.
  • MQUAKE: A rigorous multi-hop reasoning dataset requiring the linking of multiple facts (e.g., if Fact A is edited, does the model correctly update the conclusion of a chain A -> B -> C?).

Key Results and Insights

InstructEd achieved state-of-the-art (SOTA) results across LLaMA-1, LLaMA-2, and GPT-2 XL architectures. Notable performance gains include:

  • Enhanced Portability: A 10% improvement in one-hop reasoning and a 5% improvement in multi-hop reasoning over previous best-in-class methods.
  • MQUAKE Performance: On the MQUAKE-T benchmark, InstructEd reached an accuracy of 80.62%, vastly outperforming MEMIT (30.89%) and PokeMQA (75.43%).
  • Efficiency: The method is remarkably lightweight, requiring only ~0.8M additional parameters, making it a practical "plug-and-play" solution for large-scale models.

The "aha moment" of this research is that portability is fundamentally about how a model uses information. Soft instructions provide a more robust mechanism for this than natural language ("hard") instructions or direct weight manipulation.

Limitations and Future Directions

While InstructEd excels at linear reasoning chains (hops), more complex logical, mathematical, or multi-faceted common-sense reasoning remains a challenge. Additionally, while retrieval-augmented versions were tested, the noise introduced by retrieval can sometimes hinder multi-hop performance. Future work will explore scaling these dynamics to 70B+ parameter models and Mixture-of-Experts (MoE) architectures.

Summary updated

BibTeX
@inproceedings{han2024instructed,
  author = {Xiaoqi Han and Ru Li and Xiaoli Li and Jiye Liang and Zifang Zhang and Jeff Z. Pan},
  title = {{InstructEd: Soft-Instruction Tuning for Model Editing with Hops}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.888},
}
Figure