Knowledge Graph-Driven Memory Editing with Directional Interventions

Jinhu Fu, Kun Wang, Chongye Guo, Junfeng Fang, Wentao Zhang, Sen SuEMNLP 2025

TL;DR

This paper introduces KGMET, a novel framework for editing Large Language Models (LLMs) that leverages knowledge graphs to guide the editing process. KGMET addresses the challenges of inaccurate or outdated information in LLMs by enabling more stable, aligned, and controllable knowledge updates, particularly for complex multi-hop reasoning tasks, while preserving the model's general capabilities.

What Problem Does This Paper Address?

Large Language Models (LLMs) are incredibly powerful, but they often suffer from outdated or incorrect information embedded within their parameters. While model editing techniques aim to fix this by modifying specific model parameters, existing methods face significant hurdles. Firstly, many approaches struggle with "multi-hop" problems – scenarios requiring reasoning across multiple pieces of information – leading to inaccurate edits. Secondly, directly altering model parameters can inadvertently corrupt existing knowledge or lead to "catastrophic forgetting," where the model loses previously learned information. This paper tackles these issues by proposing a more robust and targeted approach to LLM editing.

Methods

KGMET introduces a three-pronged approach to knowledge editing:

  • Knowledge Graph-Guided Directional Intervention: KGMET constructs a knowledge graph from available information related to the edit request. This graph is then used to extract "directional information" from entity embeddings. This directional information guides the editing process, ensuring it targets the relevant knowledge without disrupting unrelated information.
  • Orthogonal Constraints: To prevent edits from interfering with the model's general knowledge and capabilities, KGMET employs orthogonal constraints. This technique effectively isolates irrelevant information, ensuring that updates are controllable and don't negatively impact the model's broader understanding.
  • Multi-Layer Editing: Instead of editing a single layer, KGMET extends the editing process across multiple layers of the LLM. This allows for more comprehensive and adaptable edits, making the model more capable of handling a larger number of factual requests and complex reasoning scenarios.

The core of KGMET involves constructing subgraphs from external knowledge, encoding these subgraphs using Relation Graph Convolutional Networks (RGCNs), extracting directional information from entity representations, and then applying these insights to guide parameter updates.

Datasets

The paper evaluates KGMET on three widely adopted editing datasets:

  • CounterFact: This dataset contrasts counterfactual with factual statements and is used to assess editing success and generalization.
  • ZsRE (Zero-shot Relation Extraction): This dataset is used for evaluating basic editing performance, generalization, and locality.
  • MQUAKE (Multi-hop Question Answering Knowledge Editing): This dataset is specifically designed to evaluate multi-hop reasoning capabilities after editing, requiring models to answer questions that involve reasoning across 2 to 4 hops.

Key Insights and Ideas

  • Knowledge Graphs are Crucial for Multi-Hop Reasoning: The paper strongly suggests that integrating knowledge graphs is essential for improving LLMs' ability to handle complex, multi-hop reasoning tasks after editing.
  • Directional Information is Key to Targeted Editing: Simply injecting knowledge graph embeddings can lead to "mode collapse" (where the model becomes overly focused on specific patterns). KGMET's approach of extracting "directional information" from these embeddings helps mitigate this by guiding the edits more precisely.
  • Orthogonal Constraints Preserve General Capabilities: The use of orthogonal constraints is vital for ensuring that factual edits don't degrade the model's overall performance on unrelated tasks, a common problem with other editing methods.
  • Multi-Layer Editing Enhances Scalability: Extending edits across multiple layers makes KGMET more robust and capable of handling a larger volume of edits without significant performance degradation.

Key Results

KGMET demonstrates significant improvements across various benchmarks:

  • Higher Performance: KGMET outperforms state-of-the-art (SOTA) locating-then-editing methods by an average of 27.3% on editing tasks.
  • Superior Multi-Hop Capabilities: On the MQUAKE dataset, KGMET shows substantial gains, outperforming baselines by 5-10% in neighborhood success metrics and 3-17% compared to other methods in multi-hop inference tasks. For instance, it achieved a 6-30% improvement over other baselines in multi-hop reasoning.
  • Effective Generalization: KGMET maintains the general capabilities of LLMs, showing improvements in fluency by about 20% across multiple benchmarks.
  • Scalability: KGMET remains stable and effective even when handling a large number of edits (up to 20,000 facts), outperforming other methods that show significant performance drops.

Limitations and Future Work

The authors acknowledge that KGMET's reliance on constructing knowledge graphs presents a limitation when dealing with non-structural knowledge scenarios. They suggest that exploring methods to handle such cases will be a focus of their future work.

Summary updated

BibTeX
@inproceedings{fu2025knowledge,
  author = {Jinhu Fu and Kun Wang and Chongye Guo and Junfeng Fang and Wentao Zhang and Sen Su},
  title = {{Knowledge Graph-Driven Memory Editing with Directional Interventions}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure