Scalable Model Editing via Customized Expert Networks

Zihan Yao, Yu He, Tianyu Qi, Ming Li

TL;DR

SCEN is a novel two-stage training paradigm that allows Large Language Models (LLMs) to be updated with new or corrected information without retraining the entire model. By training individual, lightweight "expert" networks for each piece of knowledge and using dedicated "indexing neurons" to trigger them, SCEN achieves state-of-the-art performance in balancing reliability, generalization to paraphrased queries, and the preservation of unrelated knowledge.

What Problem Does This Paper Address?

Large Language Models (LLMs) are often "frozen in time" based on their training data, leading to factual hallucinations and outdated responses (e.g., naming a former president as the current one). While fine-tuning can update this knowledge, it is computationally expensive and often leads to catastrophic forgetting, where the model loses previously learned information.

The field of Model Editing seeks to fix specific errors efficiently. A successful editing method must satisfy three criteria:

  1. Reliability: Does the model now give the correct answer for the edited fact?
  2. Generality: Does it still give the correct answer if the question is rephrased?
  3. Locality: Does the edit leave unrelated knowledge (e.g., the capital of France) untouched?

Existing methods often struggle with the trade-off between these three. For instance, direct fine-tuning has high reliability but poor locality, while memory-based methods can be hard to scale.

Figure 1: This is an example of model editing, where the Large Language Models (LLMs) provided an incorrect answer due to outdated knowledge (Q2). This can be corrected using model editing techniques (Q3), while ensuring a certain level of generalizability (Q4). In addition, other non-edited questions remain unaffected (Q5).

Methods

The authors introduce SCEN (Scalable Model Editing via Customized Expert Networks), a two-stage continuous training paradigm inspired by Mixture-of-Experts (MoE) architectures.

  • Stage 1: Training Experts. For every new piece of knowledge to be edited, SCEN trains a lightweight "expert" network. Specifically, it tunes the down_proj weights ($W_{down}$) of a single Feed-Forward Network (FFN) layer for that specific sample while freezing all other model parameters.
  • Stage 2: Training Indexing Neurons. To ensure the right expert is used at the right time, the authors add a "Scalable Indexing Neuron" for each expert. This neuron is trained using a specialized loss function (including margin loss) to activate only when the model encounters the specific edit sample or its semantically equivalent rewrites.

During inference, the model calculates activation values for these indexing neurons. If an activation exceeds a predefined threshold, the model swaps its original weights for the corresponding "expert" weights to generate the response.

Figure 2: SCEN Overall Architecture. The left half of the figure represents the editing stages. Stage1 is the process for training the experts and Stage2 is the process corresponding to training the indexing neurons. The right half represents the inference stage, where the corresponding experts are activated by the indexing neurons to complete the subsequent inference.

Datasets

The researchers evaluated SCEN using two primary benchmarks on Llama2-7B and 13B:

  • ZsRE (Zero-Shot Relation Extraction): A comprehensive question-answering dataset used to measure reliability, generality, and locality.
  • Hallucination Dataset: A dataset derived from WikiBio (biographies generated by GPT-3) designed to test a model's ability to mitigate factual hallucinations in long-form text.

Key Insights and Ideas

  • Granularity Matters: The authors found that assigning one expert per sample is significantly more robust than trying to force one expert to handle multiple diverse samples. When they tried "compressing" two or four samples into one neuron, performance dropped sharply.
  • Knowledge Location: By testing edits across different layers, the study confirms that factual knowledge is predominantly stored in the latter layers of the transformer architecture. Editing the 16th layer and beyond in Llama2-7B yielded much higher reliability than editing earlier layers.
  • The Threshold Trade-off: The activation threshold ($\theta$) is a critical knob. Lower values favor reliability and generality but risk damaging locality, while higher values protect original knowledge but might fail to trigger the edit.

Key Results

SCEN demonstrated superior performance across both Llama2-7B and 13B models, particularly in long-sequence editing (up to 1,000 sequential edits).

  • ZsRE Performance: On Llama2-7B with 1,000 edits, SCEN achieved an average score of 86.7% across all metrics, significantly outperforming the previous SOTA, GRACE (74.5%), and standard fine-tuning (72.7%).
  • Reliability: SCEN reached near-perfect reliability (up to 100% in some 200-edit tests), ensuring that the specific error was indeed corrected.
  • Locality: Unlike standard fine-tuning, which saw locality drop to ~38%, SCEN maintained locality scores above 83%, proving it doesn't "break" the rest of the model.
Table 1: Comparison of SCEN with existing methods. All experimental results were obtained after 200 edits.

Limitations and Future Work

While SCEN is highly effective, the authors acknowledge that adding a new neuron and expert for every single edit could eventually lead to increased storage requirements and slower inference as the number of edits grows into the thousands or millions.

Future Directions: The team plans to explore advanced weight compression techniques to reduce the footprint of the expert networks and indexing neurons, aiming to make the system even more practical for massive-scale, real-time knowledge updates in production environments.

Summary updated

Abstract

Addressing the issues of hallucinations and outdated knowledge in large language models is critical for their reliable application. Model Editing presents a promising avenue for mitigating these challenges in a costeffective manner. However, existing methods often suffer from unsatisfactory generalization and unintended effects on non-edited samples. To overcome these limitations, we introduce a novel approach: Scalable Model Editing via Customized Expert Networks (SCEN), which is a two-stage continuous training paradigm. Specifically, in the first stage, we train lightweight expert networks individually for each piece of knowledge that needs to be updated. Subsequently, we train a corresponding indexing neuron for each expert to control the activation state of that expert. We conducted a series of experiments on the ZsRE and Hallucination benchmarks by tuning the advanced open-source LLM, Llama2, achieving state-of-theart results compared to current mainstream methods. Our code is available at https://github.com/TAL-auroraX/SCEN.
BibTeX
@inproceedings{yao2024scalable,
  author = {Zihan Yao and Yu He and Tianyu Qi and Ming Li},
  title = {{Scalable Model Editing via Customized Expert Networks}},
  booktitle = {COLM},
  year = {2024},
}
Figure