Abstract
Humans rarely learn one fact in isolation. Instead, learning a new fact induces knowledge of other facts about the world. For example, in learning a korat is a type of cat, you also infer it is a mammal and has claws, ensuring your model of the world is consistent. Knowledge editing aims to inject new facts into language models to improve their factuality, but current benchmarks fail to evaluate consistency, which is critical to ensure efficient, accurate, and generalizable edits. We manually create TAXI, a new benchmark dataset specifically created to evaluate consistency in categorical knowledge edits. TAXI contains 11,120 multiple-choice queries for 976 edits spanning 41 categories (e.g., Dogs), 164 subjects (e.g., Labrador), and 183 properties (e.g., is a mammal). We then use TAXI to evaluate popular editors' categorical consistency, measuring how often editing a subject's category appropriately edits its properties. We find that 1) the editors achieve marginal, yet non-random consistency, 2) their consistency far underperforms human baselines, and 3) consistency is more achievable when editing atypical subjects.
Introduction: The Problem of Isolated Facts
Knowledge editing is the process of surgically updating specific facts within a Large Language Model (LLM)—such as correcting an outdated world leader or fixing a hallucination—without the prohibitive cost of retraining the entire model. However, current evaluation methods suffer from a significant blind spot: they primarily measure Edit Success (whether the model's output changed for the specific prompt provided) while ignoring Consistency (whether the model understands the downstream implications of that change).
In human reasoning, facts are interconnected. If you are told a "Korat" is a type of cat, you immediately infer it has claws, fur, and four legs. Without this consistency, model editing is akin to "emptying the ocean with a spoon"—fixing a single leaf on the knowledge tree while the branches remain broken. To address this, we introduce the TAXI (TAXonomic Inference) benchmark to measure how well LLMs integrate new categorical information into their internal world models.
The TAXI Benchmark
TAXI is a manually curated dataset designed to test taxonomic reasoning. It consists of 11,120 multiple-choice queries covering 976 unique edits across 41 categories (e.g., Dogs, Instruments, Foods) and 183 specific properties.
To evaluate performance, we utilize three core metrics:
- Edit Success: Did the model accept the new category (e.g., "A cobra is a dog")?
- Consistency: Did the model correctly adopt properties of the new category (e.g., "Does a cobra bark?")?
- Invariance: Did the model correctly retain properties shared by both the old and new categories (e.g., "Is a cobra an animal?")?
Evaluating Knowledge Editors
We evaluated three prominent editing approaches on the Llama-2 7B model:
- Fine-tuning (FT): Directly updating the model's weights on the new fact.
- ROME (Rank-One Model Editing): A method that treats the model's MLP layers as a key-value database, attempting to "write" a new association into a specific location.
- ICE (In-Context Editing): Providing the new fact as a prefix or instruction in the prompt (e.g., "Imagine that a cobra was a kind of dog...").
Key Results: The Consistency Gap
Our evaluation revealed a stark "consistency gap" between a model's ability to parrot an edit and its ability to reason with it.
The "Edit Success" Illusion
While ICE and Fine-tuning achieved near-perfect Edit Success (100% and 98% respectively), their consistency scores were significantly lower. ICE led with 47% consistency, ROME followed at 43%, and Fine-tuning performed no better than random chance at 23%. This suggests that models are often just repeating the edit rather than integrating it into their internal logic.
The Atypicality Advantage
Interestingly, it is easier for models to consistently edit "atypical" or rare subjects than common ones. For example, it is easier to convince a model that a rare dog breed is a cat than it is to convince it that a Labrador is a cat. This suggests that deeply ingrained "common sense" knowledge creates a form of "knowledge inertia" that is difficult to override consistently.
Human-Model Performance Gap
To establish a gold standard, we conducted a study with 19 human participants. Humans achieved approximately 87% consistency on the same tasks where the best-performing model (ICE) reached only 47%. This highlights a massive area for improvement in automated editing techniques.
Conclusion and Future Work
TAXI demonstrates that current knowledge editing techniques fail to achieve human-like consistency, particularly when dealing with well-established knowledge. While In-Context Editing (ICE) shows the most promise, it may rely on trivial shortcuts rather than true reasoning.
Future work should focus on developing editing techniques that do not just "patch" a single string of text but instead propagate changes through the model's underlying relational graphs and intuitive theories. The TAXI benchmark provides the necessary framework to measure progress toward this next generation of more robust, consistent model editors.
Summary updated