Editing Memories Through Few Targeted Neurons

Wei Zhou, Wei Wei, Guibang Cao, Fei WangAAAI 2025 · OpenReview

Abstract

Model editing is a novel research topic in large language models (LLMs), aimed at efficiently handling various knowledge editing tasks. Since irrelevant knowledge is difficult to measure, existing editing methods often lack explicit ways to preserve it, especially for editing methods based on the fine-tuning paradigm. They generally control the locality performance of model editing by constraining the range of changes in model parameters. However, their performance improvements are not always ideal, and may even lead to a decrease in the editing reliability. In this paper, we try to explore effective editing locality control methods based on the relationship between the stored knowledge and the strongly associated model components. Based on the discovery of "knowledge neurons" and enough experimental results, we further explore the potential characteristics between knowledge and model components, confirm and point out: (1) only 1% neurons have significant contributions to specific knowledge storage, and (2) these targeted neurons often have a high overlap for knowledge with similar relational descriptions, which means that knowledge with similar relationships may be severely affected when these targeted neurons are modified. Based on these findings, we propose Targeted Neurons Fine-tuning with Data Augmentation (TNF-DA), which performs data augmentation based on the relational representation of edited knowledge to improve editing locality. By freezing most of the model parameters and only fine-tuning the highly contributing neurons corresponding to the edited knowledge, we obtain desirable results in terms of generalization and specificity compared with previous fine-tuning-based methods. Extensive experiments have demonstrated the superior editing performance achieved by our proposed method.

Introduction

Large Language Models (LLMs) function as massive, static knowledge bases. However, as the world evolves—with new leaders elected and products launched—these models require updates. "Model editing" aims to modify specific facts without the prohibitive cost of retraining the entire model. The primary challenge in model editing is balancing Generalization (the model's ability to recognize the updated fact in varied contexts) and Locality (ensuring that editing one fact does not inadvertently corrupt unrelated knowledge).

Traditional fine-tuning methods often struggle with this balance, frequently leading to overfitting or "catastrophic forgetting" where unrelated facts are degraded. This research explores a more surgical approach by identifying the specific model components responsible for individual facts.

The 1% Rule: Identifying Knowledge Neurons

A core discovery of this work is that factual knowledge is surprisingly sparse within LLMs. By utilizing Causal Mediation Analysis (CMA) and Causal Tracing, the researchers identified "targeted neurons" within the Feed-Forward Networks (FFNs) that contribute most significantly to specific facts.

Figure 4: The Indirect Effect (IE) of neurons shows that only a tiny fraction (roughly 1%) of neurons contribute significantly to specific knowledge storage.

As shown in the analysis of GPT2-medium, only about 1% of neurons exhibit a high Indirect Effect (IE) for a given fact. Modifying these specific neurons is far more efficient and less disruptive than broad parameter updates. Furthermore, the study found that facts sharing the same relational description (e.g., "X was created by Y") share a high degree of overlapping targeted neurons, suggesting that neurons are often relation-specific rather than just fact-specific.

Methodology: TNF-DA

To leverage these findings, the authors propose Targeted Neurons Fine-tuning with Data Augmentation (TNF-DA). The process consists of three main stages:

  1. Locating Targeted Neurons: Identifying the 1% of highly contributing neurons in the FFN layers for the target fact.
  2. Targeted Fine-tuning: Freezing the vast majority of the model and only updating the parameters associated with these few targeted neurons.
  3. Relational Data Augmentation (DA): To preserve locality, the model is trained on a small dataset of related subjects that share the same relation. A KL-divergence loss is employed to ensure the model's performance on these related-but-unedited facts remains stable during the edit.
Figure 3: Overview of the TNF-DA framework, including (a) identifying targeted neurons, (b) analyzing relational overlap, and (c) the fine-tuning process with data augmentation and KL-divergence loss.

Experimental Results

The researchers evaluated TNF-DA using the COUNTERFACT benchmark on GPT2-XL (1.5B) and GPT-J (6B) models. TNF-DA was compared against several baselines, including standard Fine-Tuning (FT), MEND, and ROME.

  • Superior Specificity: TNF-DA achieved a Specificity score of 79.42% on GPT2-XL, significantly outperforming standard fine-tuning (30.72%) and MEND (37.90%).
  • High Generalization: It maintained a Generalization score of 87.21%, indicating the model understood the underlying concept of the edit rather than just memorizing a string.
  • Competitive Performance: With an overall Editing Score of 87.65, TNF-DA is competitive with state-of-the-art methods like ROME (90.99) while remaining conceptually simpler as a fine-tuning variant.

Conclusion and Future Work

TNF-DA demonstrates that precise model editing is possible by focusing on a tiny fraction of "knowledge neurons." While highly effective, the method currently faces challenges in reasoning—updating a fact doesn't automatically update the model's ability to use that fact in complex logic chains. Additionally, as models scale to hundreds of billions of parameters, the computational cost of causal tracing increases. Future research will explore the mutual interactions between targeted neurons and the potential for "silencing" negative neurons to improve base model performance.

Summary updated

Abstract

Model editing is a novel research topic in large language models (LLMs), aimed at efficiently handling various knowledge editing tasks. Since irrelevant knowledge is difficult to measure, existing editing methods often lack explicit ways to preserve it, especially for editing methods based on the fine-tuning paradigm. They generally control the locality performance of model editing by constraining the range of changes in model parameters. However, their performance improvements are not always ideal, and may even lead to a decrease in the editing reliability. In this paper, we try to explore effective editing locality control methods based on the relationship between the stored knowledge and the strongly associated model components. Based on the discovery of ``knowledge neurons'' and enough experimental results, we further explore the potential characteristics between knowledge and model components, confirm and point out: (1) only 1% neurons have significant contributions to specific knowledge storage, and (2) these targeted neurons often have a high overlap for knowledge with similar relational descriptions, which means that knowledge with similar relationships may be severely affected when these targeted neurons are modified. Based on these findings, we propose Targeted Neurons Fine-tuning with Data Augmentation (TNF-DA), which performs data augmentation based on the relational representation of edited knowledge to improve editing locality. By freezing most of the model parameters and only fine-tuning the highly contributing neurons corresponding to the edited knowledge, we obtain desirable results in terms of generalization and specificity compared with previous fine-tuning-based methods. Extensive experiments have demonstrated the superior editing performance achieved by our proposed method.
BibTeX
@inproceedings{zhou2025editing,
  author = {Wei Zhou and Wei Wei and Guibang Cao and Fei Wang},
  title = {{Editing Memories Through Few Targeted Neurons}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=fyJV4JyGMv},
}
Figure