Is the "Outdated Trap" a Sampling Problem or a Missing Skill?

Retrieval-augmented language models fetch documents from a search index and answer from them. They have a well-documented failure. When the retrieved text is noisy, partial, or misses the answer, they fall back on the facts they memorized during pre-training, and those facts are often out of date. RealTime QA, which poses weekly questions about current news, documented this "outdated trap." SituatedQA, which tests questions whose answers depend on time and place, found that models given fresh Wikipedia passages were still 15 points less accurate on changed facts than on stable ones. A separate literature asks whether reinforcement learning with checkable rewards teaches models anything new, or only makes them more likely to sample what they could already produce. Its answer is mostly the latter: the gains are sharper sampling, not new ability (Yue et al.; Ma and Anand).

This proposal brings that diagnostic to the outdated trap. The question is concrete. A model reads a pile of retrieved passages, and its memory disagrees with the newest evidence, or the evidence is missing altogether. Can the untrained model ever produce the right behavior if we let it sample many times? That right behavior is either the current answer or "unanswerable." If it can, cheap reinforcement learning should fix the trap. If it cannot, reinforcement learning will fail, and the model has to be taught the behavior directly with a small number of carefully chosen supervised examples. We expect the answer differs by situation, and we propose to measure it and to train accordingly.

Core Claim

We claim that the outdated trap splits into two failures with different causes. When current evidence is present but buried among distractors, the base model often can pick it out; it simply does not do so on its first try. This is a sampling-efficiency gap, and reinforcement learning with a verifiable reward should close it. When current evidence is absent, the right move is to abstain. Our hypothesis is that the base model almost never abstains, even across hundreds of samples, because nothing in pre-training rewards a model for declining to state a fact it confidently remembers. This is a ceiling. The pass@k literature predicts that reinforcement learning cannot break it. Supervised examples that cannot be solved by either sub-skill alone should break it: ignoring distractors is not enough, and trusting the context is not enough.

The new elements are three. The first is a pass@k ceiling analysis applied to choosing between memory and context, rather than to math or code. The second is a reward that can tell a stale answer apart from an ordinary wrong one. SituatedQA records answers at several dates, so "the model gave the 2018 answer" can be checked automatically. The third is a training recipe whose two halves are each justified by that diagnosis.

Research Program

We first build a controlled test grid from the time-dependent questions in SituatedQA and from post-cutoff weeks of RealTimeQA. For each question we construct contexts along two axes. The evidence axis has three conditions: a passage with the current answer is present, only a passage with an older answer is present, or no answer-bearing passage is present. The noise axis adds retrieved distractor passages at $k \in \{1, 5, 20, 50, 100\}$. These levels are chosen to span the inverted-U relationship between context size and accuracy reported by LDAR and the collapse at 100 documents reported by Cho et al.. RealTimeQA weeks after a model's training cutoff give questions where the current fact is truly absent from its weights. In those cases a "stale" answer can only come from older memorized facts. That is the knowledge-ceiling setting in its purest form.

We sort every output into four classes: current answer, stale answer (matching an earlier dated answer), other wrong answer, and abstention. For the base model we estimate, in each cell of the grid, the probability that at least one of $k$ samples falls in the correct class. The correct class is the current answer when evidence is present and abstention when it is absent. We use $k$ up to 1024, following the protocol of Yue et al.. This map is itself a deliverable. It shows, cell by cell, whether the right behavior is latent in the model.

We then train with reinforcement learning (GRPO) under a reward that separates the four classes:

$$r = \begin{cases} 1 & \text{current answer, evidence present} \\ 1 & \text{abstention, evidence absent} \\ -\lambda & \text{stale answer} \\ 0 & \text{otherwise} \end{cases}$$

Here $\lambda \ge 0$ sets how much worse a stale answer is than a generic error. With $\lambda=0$ this reduces to ordinary accuracy-plus-abstention reward. In parallel, we build a small supervised set in the spirit of the "irreducible" examples of Ma and Anand. Each example has three properties at once: a confident memorized answer, distractors at high $k$, and either superseding evidence or no evidence. The correct output therefore requires rejecting memory and ignoring noise together. The matched "reducible" control set contains the same number of examples, but each exercises only one of these skills. We train on supervised data first and then reinforcement learning, as the ceiling literature recommends. Models are Qwen3-8B, the model used by Ma and Anand, and Llama-3.1-8B-Instruct, the model used by Ext2Gen.

Baselines

The naive reference is the instruction-tuned model reading all $k$ retrieved passages with a standard RAG prompt, together with the same model closed-book. This anchors the stale rate. Kasai et al. report that GPT-3 reached 66.5% multiple-choice accuracy on RealTimeQA with live search, against 44.9% closed-book and 43.9% with a 2018 Wikipedia index (RealTime QA). We do not claim those numbers transfer to our models; we rerun the reference ourselves.

The strongest existing competitor is Chain-of-Note. It fine-tunes a 7B model on 10k GPT-4 traces that judge each passage's relevance before answering. It reports a +10.5 gain in correct rejection rate on RealTimeQA and +7.9 EM when all passages are irrelevant. It is the closest work because it targets both noise and abstention with supervised training. We will retrain its recipe on our backbones with the same base model, the same 10k-example data budget, and matched GPU-hours. Beating it means a lower stale rate and higher correct abstention in the evidence-absent cells, at equal or better accuracy in the evidence-present cells. We also aim for an inference cost far below its reported 12.0s per query, because our model emits no notes.

The strongest training-free competitor is Distraction-aware Answer Selection (Cho et al.). It adds an explicit "return unanswerable" instruction and reranks answers by how likely each passage makes the query. It reports a 64% average relative EM gain and 231% on SQuAD with 100 DPR passages. We run it on the same grid with the same sampling budget. For temporal routing we compare against the fact-duration method of Zhang and Choi. It predicts how long a fact lasts and triggers retrieval only when the fact has likely expired, and it reports 20.5 EM on SituatedQA while retrieving for 45% of queries. That result comes from only 322 examples, so we rerun it on our larger grid.

On the mechanistic side we include ContextFocus, which adds a steering vector at inference time to push the model toward the context and away from memory. We predict it lowers the stale rate but raises the rate at which the model adopts distractors at high $k$. Our ablations isolate each ingredient: reinforcement learning with $\lambda = 0$; reinforcement learning alone without the supervised stage; and irreducible versus reducible supervised sets of equal size. No reported result was found on Lacuna for reinforcement learning with verifiable rewards on temporal RAG, so every reinforcement learning number here will be new.

Why not just…?

Why not just tell the model to answer only from context and abstain otherwise? The negation prompt in DAS does exactly this and helps under noise. But Zhang and Choi report that prompt-level abstention in GPT-4 over-abstains on answerable questions. ContextFocus finds prompting alone does not overcome strong memorized priors. Our grid measures both failure directions: false abstention in evidence-present cells and stale answers in evidence-absent cells. That makes the trade-off visible rather than assumed.

Why not just run standard reinforcement learning on RAG question answering, without the diagnosis or the stale penalty? This is the real threat. If GRPO with an ordinary accuracy-plus-abstention reward matches our full recipe, the supervised stage and $\lambda$ are unnecessary. The $\lambda = 0$ ablation and the reinforcement-learning-only ablation settle this directly. Our prediction rests on the crossover findings of Yue et al. and Ma and Anand. They predict that plain reinforcement learning raises first-try accuracy in evidence-present cells but leaves large-$k$ abstention in evidence-absent cells at the base model's level. If that prediction fails, the result still matters, because it would show that refusing stale memory is latent rather than missing.

Why not just mute the model's memory, as ParamMute or PH3 do? ParamMute suppresses feed-forward layers that inject memorized facts. PH3 prunes attention heads associated with memory. Both were tested on clean counterfactual conflicts. Muting memory does not teach abstention. When the context lacks the answer, a context-only model should latch onto distractors instead of stale facts, which trades one error for another. The evidence-absent, high-$k$ cells test exactly this prediction.

Why not just compress or filter the context first? Ext2Gen already shows that joint extraction beats standalone compressors (0.557 vs 0.473 on Llama-3.1-8B). However, filtering only addresses noise. It cannot tell the model what to do when the surviving passage is outdated or absent, which is the cell this proposal targets.

Decisive Checks

The central test is the large-$k$ crossover in the evidence-absent cells. The ceiling hypothesis predicts three outcomes there. The reinforcement-learning-only model's pass@1024 for abstention should not exceed the base model's. The irreducible-supervision model should raise it substantially. The reducible control of equal size should not. The second check is the stale-specific penalty: at matched overall accuracy, $\lambda > 0$ should lower the share of errors that are stale answers relative to $\lambda = 0$, measured on RealTimeQA weeks after the training cutoff. The third check is that gains hold across every noise level. Any method that improves the $k=1$ cells while losing ground at $k=100$ has only moved the noise problem elsewhere. We report all four output classes per cell, so a reader can see exactly which failure each method removes and which it creates.