When a question needs several lookups, such as "Where did the current CEO of Company X go to university?", a retrieval system first resolves "the CEO of X" and then asks a second question about that person. If the first lookup returns a CEO who left two years ago, the second lookup still works. It finds that person's university, cleanly and confidently, and the system gives a fluent wrong answer. Nothing in this chain looks like noise: the documents are on topic, the reader is not distracted, and every step "succeeds." The only error is that one intermediate fact has expired. Current defenses miss it. Noise filters look for off-topic text. Staleness detectors check only the final answer. Planners that break a question into a graph of sub-questions fix the graph before running it.
We propose borrowing a mechanism from database engines. A database engine re-plans a query mid-execution when an intermediate result breaks the assumptions the plan was built on. Our version attaches an expiry guard to every intermediate answer in a multi-hop retrieval plan. The guard estimates, from how long facts of that kind usually last and how old the supporting evidence is, whether the fact is still valid at question time. When a guard fires, only that node is re-executed against a fresher source, and only its downstream sub-questions are re-run. Results from stable branches are cached and reused. The claim is that obsolescence spreads through multi-hop plans, so it should be caught at the hop where it enters rather than at the end.
Core Claim
Temporal distractors are documents about the right entity but from the wrong time. In multi-hop question answering they behave differently from topical distractors. A stale binding at hop $k$ turns every later hop into a well-posed question about the wrong entity, so later retrievals look more confident, not less. Guarding each intermediate binding with a fact-lifespan check, and re-planning only the affected sub-graph, should recover most of these errors. It should also do so with far fewer calls to a live source than routing every hop to live search.
The guard is the lifespan model of Mitigating Temporal Misalignment by Discarding Outdated Facts. That work trains a BERT-base classifier to predict how long a stated fact will hold, as a distribution over 13 duration bins. It then lowers answer confidence when the gap between query time and evidence time is too large, but only for single-hop answers and only as a retrieve-or-not switch. We reuse their classifier as a per-node invariant. Let $F_r$ be the predicted cumulative distribution of lifetimes for relation statement $r$, let $t_E$ be the evidence date (or an extracted start date such as "since 2019" when the text gives one), and let $t_q$ be the question date. The node's survival probability is
$$S = 1 - F_r(t_q - t_E),$$
and the guard fires when $S < \tau$, for a threshold $\tau$ tuned on held-out data. This works like the logical validity regions that Efficient Adaptive Detection of Complex Event Patterns attaches to plan fragments, where re-optimization fires only when a stated constraint is violated. Here the constraint is temporal validity rather than relative table size.
Research Program
The plan representation follows Plan*RAG. That system decomposes a question into a directed acyclic graph of atomic sub-queries, each answered by a single retrieval, with parent answers filled into child slots. Its authors explicitly leave verification and backtracking open. We change two things.
The first change is what each node stores. Each node keeps a small record (binding, evidence date, extracted start date if any, survival score), not a bare string. This is the analogue of the materialization checkpoint in How I Learned to Stop Worrying and Love Re-optimization: there, an intermediate result becomes a relation of known size before the optimizer re-plans the rest of the join tree.
The second change is what happens when a guard fires. The executor takes one of three actions, chosen in a fixed order. It first reroutes the node from the frozen local index to a live search operator, with an explicit "as of $t_q$" anchor. This swaps a data source the way SQL Anywhere swaps a join operator when memory changes. The executor then invalidates and re-runs only that node's descendants. If the live source also fails to produce evidence dated after the old binding, the node is marked unknown and the system abstains, rather than letting the reader fill the gap from its weights. That last action targets a specific finding. RealTime QA reports that retrieval insufficiency caused 34% of errors, and that when retrieval failed, models fell back on outdated training knowledge instead of abstaining.
Re-executed nodes are answered by extraction only, using a span from the fresh evidence. This follows the extract-then-generate idea in Aligning Extraction and Generation for Robust Retrieval-Augmented Generation. The motivation is the "Update Fallacy" in SituatedQA: even with an updated corpus, models were 15 points less accurate on changed facts than on stable ones, because stale parametric memory competes with the context. Fetching fresh text is not enough if the reader is then free to override it.
Two parts of the literature shape how much we should expect. Database work found that re-planning pays off mainly for deep join trees: perfect estimates on joins of 2 or 3 tables gave almost no speedup (Perron et al.). Multi-hop questions are shallow. We therefore predict that the benefit grows with the number of hops downstream of the stale node, and we test that prediction directly instead of assuming it. Separately, Stop-RAG shows that the cost of extra context depends on how well the reader tolerates noise. Our guard avoids that problem because it decides from evidence dates, not from estimated reader robustness.
Baselines
The naive reference is a static plan. We run PlanRAG-style decomposition and execution over the same frozen index with the same reader, no guards, and no rerouting. This measures how much accuracy stale chains cost. No reported result was found on Lacuna for PlanRAG on the temporal benchmarks we use, so we run it ourselves.
The strongest existing method is Review-Then-Refine. It decomposes questions sequentially, rewrites each sub-query with a temporal anchor, gates retrieval on an LLM judgment, and runs a contextual contradiction check before answering. It is the strongest competitor because it already targets temporal multi-hop questions with adaptive retrieval, and it reports 60.60% accuracy on FreshQA (versus 50.01% for DSPy) and 27.59% on PAT-Questions. To make the comparison fair, we give it the same reader model, the same local index, and the same live search operator. We match total LLM calls and total live-search calls within 10%. We also tune its prompts with the same number of development trials we spend tuning $\tau$. Beating it means higher accuracy on FreshQA and PAT-Questions at equal or lower live-search budget, plus a significant gain on the stale-chain diagnostic described below.
We also include final-answer gating: the lifespan-based confidence decay of Zhang and Choi, applied only to the final answer, which triggers a full re-run with live search. They report 20.5 EM on SituatedQA while retrieving for 45% of queries. This baseline uses the same duration model in the place prior work puts it.
Three ablations isolate our key ingredient. The first keeps per-node guards but re-runs the whole plan instead of only the descendants. The second replaces the lifespan guard with an LLM self-assessment of staleness at each node. The third keeps guards and rerouting but lets the reader answer freely, without extraction-only answering.
Why not just…?
Why not just route every hop to live web search? This is the most serious threat. RealTime QA shows that GPT-3 with Google search reaches 66.5% multiple-choice accuracy, against 43.9% with a 2018 Wikipedia dense index. If live search were free and unbiased, always using it could match our accuracy. We report an accuracy-versus-live-calls curve with "always live" as one endpoint. Our claim holds only if guarded rerouting reaches within one point of always-live accuracy using a small fraction of the live calls, since most hops (birthplaces, founding dates) are stable. Even always-live search does not address the Update Fallacy, so the extraction-only ablation tests whether the reader still overrides fresh evidence.
Why not just rerank retrieved passages by recency? When every document carries a reliable date and an update has been written, keeping the newest passage about the entity may solve much of the problem. That is a real competitor, and we include it as a strong baseline. It fails in two cases: when the frozen index contains no post-change document at all, which is the RealTime QA retrieval-insufficiency regime, and when the newest local document about an entity is itself stale. Only the lifespan check notices that a recent-looking record has probably expired. The diagnostic set reports these strata separately. If recency reranking matches us on all strata, we will report that the guard adds nothing beyond it.
Why not just let an adaptive agent decide at each step, as Review-Then-Refine does? Its retrieval gate asks the model whether its own knowledge suffices. That is exactly the judgment the Update Fallacy says is unreliable, because the model is confident about stale facts. The LLM-self-assessment ablation tests this directly. Our prediction is that LLM staleness judgments miss stale bindings on facts the model is confident about. If the ablation matches the lifespan guard, the duration model is unnecessary and we will say so.
Proposed Experiments
The decisive test is a stale-chain diagnostic. We start from the 2018 Wikipedia snapshot that Plan*RAG uses as its frozen index and pair it with a current snapshot. We then compose 2- to 4-hop questions in which a specified hop's answer changed between snapshots, recording how many hops lie downstream of the changed one. Gold answers come from the current snapshot. We stratify the questions three ways: by depth of the changed hop, by whether the frozen index contains any post-change document, and by relation volatility. If per-node guarding beats final-answer gating, and the gap widens with the number of downstream hops, then obsolescence really does compound through the plan, mirroring the depth effect in database re-planning. If the gap is flat, per-node machinery is unjustified and the simpler final-answer gate suffices.
On the public benchmarks we evaluate accuracy on FreshQA and PAT-Questions against Review-Then-Refine's reported 60.60% and 27.59%, rerun under matched budgets. We report EM and calibration error on SituatedQA against the 20.5 EM final-answer-gating result. We report RealTime QA accuracy as a function of live-call budget. To rule out regressions on static questions, where guards should rarely fire, we report F1 on MuSiQue against Stop-RAG's 47.0. Throughout, we track the guard firing rate, its precision against the snapshot diff, wall-clock latency, and LLM and search call counts.
A final check addresses the duration model's start-date blind spot. We compare guards that use evidence date only, guards that use an extracted start date, and guards that use an oracle start date. This shows how much of the remaining error comes from not knowing when a fact began, rather than from mispredicting how long it lasts.
References
- Mitigating Temporal Misalignment by Discarding Outdated Facts
- Efficient Adaptive Detection of Complex Event Patterns
- Plan$^\ast$RAG: Efficient Test-Time Planning for Retrieval Augmented Generation
- How I Learned to Stop Worrying and Love Re-optimization
- SQL Anywhere: An Embeddable DBMS
- RealTime QA: What's the Answer Right Now?
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- SituatedQA: Incorporating Extra-Linguistic Contexts into QA
- Stop-RAG: Value-Based Retrieval Control for Iterative RAG
- Review-Then-Refine: A Dynamic Framework for Multi-Hop Question Answering with Temporal Adaptability