TL;DR
When we "edit" a fact in a Large Language Model (LLM), is the old knowledge truly deleted or just buried? This paper introduces MEMENTO, a technique that recovers "deleted" pre-edit knowledge using only specialized input prompts—without ever touching the model's weights. The results suggest that current model editing techniques primarily suppress factual accessibility rather than eliminating the underlying information.
What Problem Does This Paper Address?
As LLMs become central to our information infrastructure, we need ways to update them—correcting outdated facts or removing sensitive personal information. "Model editing" (like the popular ROME or MEMIT methods) offers a surgical alternative to expensive retraining by identifying and rewriting specific weights responsible for a fact.
However, a critical question remains: Does editing actually erase the old knowledge?
Previous attempts to "undo" edits relied on weight-space interventions—essentially hacking the model's parameters back to their original state. This paper investigates whether that knowledge is still lurking "beneath the tokens," recoverable through clever prompting alone. If it is, then model editing might be a less permanent "delete" than we previously thought, posing significant implications for privacy and model safety.
Methods
The authors propose MEMENTO, a prompt-space auditing mechanism. Instead of modifying the model $f_\theta$, they learn a reusable "recovery signal" $\Pi$ that is prepended to the input.
MEMENTO comes in two flavors:
- Continuous Recovery: Learns a "soft" prefix (a sequence of trainable embeddings). This is highly effective but requires access to the model's embedding layer.
- Discrete Recovery: Uses gradient-guided search (GCG) to find a sequence of actual text tokens that act as a "skeleton key" to unlock the pre-edit knowledge. This works even through standard text-only API interfaces.
To train these signals, the authors used a multi-part objective function:
- Recovery Loss: Increases the likelihood of the original (pre-edit) answer.
- Preference Loss: Specifically pushes the model to prefer the old answer over the new, edited one.
- Fluency & Distillation: Ensures the prefix doesn't break the model's general language abilities or change its behavior on unrelated topics.
Datasets
The study utilizes the CounterFact dataset, a standard benchmark for model editing that contains thousands of factual triples (e.g., "The Eiffel Tower is in [Paris]").
- Training: 1,000 facts to learn the recovery signal.
- Evaluation: 1,000 held-out facts to test if the signal generalizes to unseen edits.
- Models: Evaluated across GPT2-XL (1.5B), GPT-J (6B), and Llama3 (8B).
- Editing Techniques: Tested against five different methods: MEMIT, AlphaEdit, PRUNE, RECT, and CoME.
Key Insights and Ideas
- Suppression, Not Deletion: The most striking insight is that editing is often a "mask" rather than an "erasure." By simply changing the input context, MEMENTO can bypass the parameter updates and elicit the original response.
- The Accessibility Gap: In a surprising twist, the authors applied MEMENTO to pristine, unedited models. They found it significantly boosted factual recall. This suggests that LLMs often "know" facts they fail to retrieve during standard inference—a phenomenon the authors call the "accessibility gap."
- Cross-Method Generalization: A recovery signal trained on a model edited with MEMIT can often recover facts from a model edited with AlphaEdit. This implies that different editing techniques might be suppressing knowledge in similar, predictable ways.
- Mechanistic Transparency: Using "logit lens" analysis, the authors showed that without MEMENTO, the original answer loses support in the later layers of the edited model. MEMENTO restores this support, allowing the original factual trajectory to re-emerge as the data flows through the network.
Key Results
- High Recovery Rate: Continuous MEMENTO recovered 80.07% of held-out edited facts across all models and techniques, a 9.58% relative improvement over the previous state-of-the-art weight-masking baseline (One Mask).
- Black-Box Auditing: The discrete (text-only) variant achieved up to 49.0% recovery, proving that "deleted" knowledge can be extracted through a standard chat interface.
- Recall Boost: On unedited models, MEMENTO improved average factual recall by 53.9% relative to standard prompting.
As seen in the results table above, MEMENTO consistently outperforms the baseline (One Mask) in almost every category, particularly on more modern architectures like Llama3.
Limitations and Future Work
- Internal Representation: While MEMENTO proves behavioral recoverability, it doesn't prove that the knowledge remains "intact" in its original internal representation. It just shows the model can still produce the answer.
- Scope of Models: The study focused on models up to 8B parameters. Whether these findings scale to 70B+ or frontier models remains an open question.
- Safety Implications: If "deleted" PII (Personally Identifiable Information) can be recovered via a prompt, current editing-based unlearning methods are insufficient for privacy compliance. Future work needs to focus on more robust "hard" deletion methods that are resistant to prompt-space recovery.
Summary updated