Remembering Where It Slipped: Failure Memory for Facts a Model Already Knows

Language models often state false things about facts they actually know. A model that answers a trivia question correctly under a plain prompt can get the same question wrong when the prompt carries a misleading persona, a false premise, or its own earlier mistake. Distinguishing Ignorance from Error in LLM Hallucinations calls these "known-fact errors" and finds they affect 4–12% of known facts under a mild nudge and up to 24% under snowballing. This proposal tests whether such slips can be fixed by giving the model a memory of its own past slips. When the model is about to answer, a probe reads its internal state. If that state looks like a situation where it slipped before, the system retrieves a few of those past failures, each with the wrong answer, the correct answer, and what misled it, and puts them in the prompt. Nothing is retrieved for questions the model simply doesn't know. Those go to abstention or document retrieval, because reminders cannot supply missing knowledge.

The two-sentence version: store only the model's own mistakes on facts it demonstrably knows, and look them up by what the model's hidden activations looked like when it failed, not by how the question is worded. The test is whether this beats three cheaper options on new questions about new entities: a generic "be factual" instruction, retrieval of similar correct examples, and layer-contrastive decoding.

Core Claim

Failure-driven reminding comes from classical case-based planning. It is the idea that a system should index past failures by their cause, so the right warning comes back in a new situation. It has a natural home in factual hallucination, but only for one class of error. CHEF: A Model of Case-Based Planning indexed plans by the failures they avoid. Failure-Driven Reminding for Incremental Learning kept generally good rules and attached exception episodes that add caution only in brittle contexts. That is exactly the structure of known-fact errors: the model's knowledge is fine, and it breaks only in specific contexts.

Three pieces of evidence point toward a particular design. The first is the Ignorance-vs-Error paper's asymmetry: prompting repairs known-fact errors (up to 51%) but barely touches ignorance (<3%). So a reminder memory should admit only known-fact failures.

The second is how little these errors overlap across models. In the same paper, the Jaccard overlap of known-fact errors between models is only 0.1–0.36. The memory must therefore be built from the deployed model's own failures, not borrowed from another model.

The third is that pre-answer internal states already carry the warning sign. The Ignorance-vs-Error paper predicts upcoming known-fact errors at 60–70% accuracy from the residual stream (the running hidden vector each layer adds to) at the last prompt token. On Large Language Models' Hallucination with Regard to Known Facts reaches 88% from how the correct token's probability evolves across layers. Our claim is that these internal signatures work better as memory keys than question text. Two questions about unrelated entities can share a failure mode, such as deference to a stated persona or anchoring on an earlier wrong claim. Surface-similarity retrieval misses that link. Activation-similarity retrieval might not.

Relation to Prior Work

Several systems already bring failure memory into language models. We depart from each in a specific way.

Reflexion: language agents with verbal reinforcement learning turns task feedback into written self-critiques and retries the same problem. It is intra-instance and depends on an execution environment or exact-match checker. We retrieve across queries, and our failure signal comes from the model's own disagreement with itself rather than self-reflection, which Reflexion itself notes can produce hallucinated rationalizations.

Memory-assisted prompt editing to improve GPT-3 after deployment retrieves past user corrections across queries, keyed by a generated paraphrase of the query intent. It depends on a human or simulated user and targets misunderstood instructions rather than factual slips. We supply the supervision and change the key.

The episodic memory condition in Evaluating Long-Term Memory for Long-Context Question Answering prepends free-text reflections on past errors in long-dialogue QA. It helps with unanswerable questions but never separates error types and never touches parametric knowledge.

Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning shows that error-aware rethinking helps on math and logic. It uses generic error categories rather than retrieved episodes. Together, these systems supply our method template, and none addresses facts the model knows but misreports.

The mechanistic probing papers play a different role. They supply our gate and our key, but they stop at diagnosis.

The decoding and fine-tuning methods are our competitors. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models and HaDeMiF: Hallucination Detection and Mitigation in Large Language Models adjust token scores statelessly. GRATH: Gradual Self-Truthifying for Large Language Models fine-tunes on self-generated correct/incorrect pairs and throws the pairs away afterward. It also collapses when iterated beyond two rounds, which is one constraint motivating a non-parametric store whose entries can be added and deleted.

Finally, Adaptive Memory Admission Control For LLM Agents shows that deciding what enters memory matters. It found that removing a type prior cost 0.107 F1 on LoCoMo, a benchmark for long-term conversational memory. Our admission rule is a mechanistic type prior of that kind.

Research Program

The system has four parts: labeling, admission, gating, and retrieval.

Labeling. For each question in a memory-building split, we follow the WACK protocol from the Ignorance-vs-Error paper. The model is sampled under a neutral prompt to decide whether it knows the fact, then queried under nudges: persona, false premise, and snowballed prior claims. A wrong answer under a nudge, on a fact the model answers correctly when neutral, is a known-fact failure.

This labeling largely removes the need for gold labels. The model's own neutral answer serves as the reference, which partly addresses the "flaky evaluator" problem that limits Reflexion outside code. We run a gold-label version and this self-referenced version side by side and report both.

Admission. Only known-fact failures enter memory. Each entry stores four things: the last-prompt-token residual vector at a chosen layer, the question, the wrong answer, and the correct answer. It also stores a short cause tag produced by comparing the nudged and neutral prompts. For example, "deferred to the stated persona" or "repeated a claim from its earlier turn."

Gating. At test time, a linear probe trained on the same split scores the incoming prompt's hidden state. When the score exceeds a threshold, the system retrieves the $k$ entries whose stored vectors have the highest cosine similarity to the current one. It prepends them as short contrastive reminders ("Earlier, in a similar situation, you answered W; the correct answer was C; you were misled by …"). A separate knowledge probe, trained to recognize ignorance, routes likely-unknown questions to abstention or Wikipedia retrieval instead.

We will study how the probe's layer and the threshold trade accuracy against added prompt tokens. We will also measure memory growth: does performance keep improving, plateau, or degrade as the store reaches thousands of entries, as MemPrompt-style memories risk?

Baselines

The naive reference point is the same model under the same nudged prompts with no intervention. On TruthfulQA, GRATH reports Llama2-Chat-7B at 30.23% MC1 and 45.32% MC2. (TruthfulQA tests whether models repeat common human misconceptions. MC1 scores picking the single true option, and MC2 scores the probability mass the model puts on the true options.) We use Llama2-Chat-7B as the primary model so this and other reported numbers are comparable. We add a second open model because error sets differ across models.

The strongest cheap competitor for known-fact errors is the "be factual" instruction. In the Ignorance-vs-Error paper it reduced known-fact errors by up to 51% on the WACK setup. We run it with the identical nudges and questions and tune its wording on the memory split with the same number of trials our method gets. Beating it means a statistically reliable further reduction in known-fact error rate on held-out entities. Since "be factual" can be stacked on top of our reminders, we also report the combination.

The strongest retrieval competitor is nearest-neighbor few-shot prompting. It retrieves the $k$ most text-similar questions from the same memory split, with correct answers only and no failure framing, and uses the same $k$ and token budget. This baseline isolates whether the gain comes from remembering failures or simply from showing relevant correct examples.

The strongest decoding competitor is DoLa, run from the official code (voidism/DoLa) with its default layer buckets. The DoLa paper reports gains of 17 absolute points for LLaMA-7B and over 12 for LLaMA-65B on TruthfulQA. Because DoLa and our method act at different stages, we also test DoLa plus reminders.

The strongest weight-updating competitor is GRATH, which reports Llama2-Chat-7B at 54.71% MC1 and 69.10% MC2. We compare against this reported number and, where compute permits, rerun GRATH at two iterations with pairs drawn from our memory split so both methods see the same questions. Matching GRATH without weight updates, while preserving performance on the neutral prompts, would count as a meaningful result. We do not expect to exceed it on TruthfulQA. This is because imitative falsehoods overlap only partly with nudge-induced slips, a point we test rather than assume.

We also run four ablations that isolate the key ingredient:

  • text keys versus hidden-state keys, with everything else fixed;
  • admitting all failures versus only known-fact failures;
  • contrastive reminders versus correct-answer-only reminders;
  • gated versus always-on retrieval;

plus a random-entry memory to control for the mere presence of extra text.

Why not just…?

Why not just tell the model to be factual? The Ignorance-vs-Error paper shows this instruction already repairs a large share of known-fact errors, so it is a real threat. The instruction is global, however. It cannot tell the model which contexts have fooled it before. And snowballing, where error rates climb to 24%, is exactly the case where a model's own earlier text overrides a generic instruction. The decisive comparison is on the snowballing and persona nudges with held-out entities. If "be factual" matches reminders there, the memory adds nothing and we will say so.

Why not just retrieve similar solved examples? Showing the model correct answers to nearby questions may explain any gain from a failure memory, especially if test questions paraphrase memory questions. We guard against lookup by splitting on entities, so no test entity appears in memory, and on TruthfulQA categories, holding out whole categories of its 38. If correct-only text-nearest-neighbor retrieval matches our method under these splits, the failure-driven framing is unnecessary and the contribution shrinks to a probe-gated retriever. We treat that outcome as a real possibility.

Why not just use document retrieval from Wikipedia? Retrieval addresses missing knowledge, which our system deliberately routes to retrieval anyway. It does not address a model overriding knowledge it has. Even search-grounded PerplexityAI scores only 71.5% on FActScore biographies in FActScore; FActScore splits long answers into atomic claims and checks each against Wikipedia. We include retrieval-only as a baseline on the WACK nudged set. If retrieved passages eliminate nudge-induced slips as well as reminders do, that settles it against us.

Why not just fine-tune on the failures? GRATH shows that preference tuning on self-generated pairs works but degrades past two iterations. It also cannot delete a bad update without retraining. Our experiment compares the two directly on shared questions and adds a sequential setting in which failures arrive in batches. In that setting the memory is updated after each batch and GRATH must be retrained.

Proposed Experiments

The primary benchmark is the WACK setup of the Ignorance-vs-Error paper on TriviaQA and Natural Questions, with its four nudges. It is split by entity into memory-building, probe-training, and test portions. The main metric is the known-fact error rate on the test portion. It is reported separately for each nudge and alongside the ignorance error rate, which our method should leave unchanged except through routing.

Secondary evaluations are TruthfulQA MC1/MC2 and open generation with category-held-out splits, and FActScore biographies, with ChatGPT's reported 58.3% as a reference. FActScore tests whether reminders transfer from short answers to long generation, where the paper shows factual precision decays.

We also report the added prompt tokens per query and the fraction of queries that trigger retrieval, and we check accuracy on neutral prompts to confirm the reminders do not induce over-caution.

The decisive check is a single comparison on the snowballing nudge with held-out entities: hidden-state-keyed, known-fact-only, contrastive reminders against text-keyed correct-example retrieval and the tuned "be factual" prompt. The central hypothesis succeeds if it wins that comparison and its gain vanishes on ignorance items. It fails outright if text-keyed correct examples do as well. A second decisive check asks whether the self-referenced, label-free memory retains most of the gold-label memory's gain. If it does, this is a failure-driven memory for open-domain facts that needs no external checker.