Conversational search systems have to turn a follow-up like "who coaches them now?" into a standalone query a search engine can use. One of the most effective tricks for this has a counterintuitive core: let a language model guess the answer and add the guess to the query. A wrong guess still tends to contain the right kind of words, and those words help retrieval. That trick carries a hidden risk. For facts that change over time, the model's guess is not random noise. It is the answer that was true when the model was trained. Adding that guess steers the retriever toward documents about the past, and the downstream answer comes out confidently out of date.
We propose to measure this failure and fix it with a small, targeted mechanism. A per-turn gate estimates whether the model's guessed answer has probably expired. The gate is built from a predictor of how long a fact of that type usually stays true, paired with an estimate of when the fact last changed. When the guess has likely expired, the system keeps the useful vocabulary of the guess, meaning answer type, relation, and topical terms, and strips the specific stale entity. The alternative in that case is to re-ground the guess on a first round of retrieval. The claim is narrow and testable. On time-sensitive conversational turns, guessed-answer expansion raises the rate at which outdated passages are retrieved, and an expiry gate removes most of that harm without losing the expansion gains on stable facts.
Core Claim and Where It Sits
The mechanism we examine comes from ConvGQR. It pairs a rewriting model with a second pretrained model that generates a "potential answer" for query expansion. The authors describe hallucination explicitly as "a powerful tool for query expansion," and the method beats human rewrites on QReCC, with MRR (mean reciprocal rank, the average of one over the rank of the first relevant passage) of 42.0 against 38.4. The later strong methods for conversational query reformulation (CQR), IterCQR, AdaCQR and ConvSearch-R1, optimize rewrites directly against retrievers through iterative risk-minimizing sampling, contrastive alignment over BM25 and dense rankings, and reinforcement learning with a reasoning step. All four are trained and evaluated on TopiOCQA, QReCC and TREC CAsT, where the facts are treated as fixed in time. None of them asks whether the entities they inject are still current.
Three findings motivate the concern. RealTime QA shows that when retrieval is weak, models fall back on outdated training facts. SituatedQA shows systems are 15 points less accurate on updated facts than on stable ones, even when current text is indexed; the authors call this the "update fallacy." DYNAMICQA shows models resist correction most on frequently edited facts (r ≈ −0.27 between edit count and persuasion).
Our hypothesis joins these findings. Guessed-answer expansion may be one concrete route by which the update fallacy enters conversational search: the stale entity reaches the query before retrieval has had any chance to correct it. This is the first thing we test. If it is false, the rest of the proposal does not matter.
The gate reuses the method of Mitigating Temporal Misalignment by Discarding Outdated Facts. That work predicts a fact's lifespan $d$ with a cloze-style classifier, compares it with the misalignment $m = t_q - t_E$ (query time minus evidence time), and uses the result to lower confidence or trigger retrieval in single-turn QA. It reports a 50–60% reduction in expected calibration error (ECE) and 20.5 exact match (EM) on SituatedQA, while retrieving for only 45% of queries. We move the idea to a different decision point. Instead of deciding whether to answer, the gate decides whether a parametric guess is safe to put into a search query. We also address a limitation that paper leaves open: it predicts lifespan without estimating when the fact began.
Two results constrain the design. DYNAMICQA finds that semantic entropy, a measure of disagreement across sampled answers, does not track persuadability on evolving facts, so we do not rely on sampling disagreement as the staleness signal. Can Agent Memory Systems Track Evolving State? finds that chain-of-thought reasoning can rationalize outdated facts instead of correcting them. That warns against assuming the reasoning step in ConvSearch-R1 already fixes the problem, so we test that assumption directly.
Research Program
The first component is a benchmark, because none of the files' conversational benchmarks contains facts that change over time. We start from the temporal questions in SituatedQA and the temporal split of DYNAMICQA, both of which come with answers tied to dates or with edit histories. We turn each question into a short conversation in the style of TopiOCQA and QReCC. An opening turn introduces the subject entity. The target turn is an elliptical follow-up whose answer changed at a known date.
The retrieval corpus mixes passages from an older and a newer Wikipedia snapshot. Passages stating the current answer are labeled relevant, and passages stating a superseded answer are labeled stale.
Two design choices make the benchmark usable. First, a turn counts as stale-prone only if the expansion model's closed-book answer actually equals a past value. This determines the model's effective cutoff by observing its behavior, rather than assuming it. Second, every temporal conversation gets a matched control conversation with the same structure about a stable fact. The comparison between the two isolates the effect of time. Conversation openers are written by a language model from templates, and a human-checked subset is used to estimate label noise.
The second component is the gate itself. For the resolved rewrite, a lifespan predictor in the style of Zhang and Choi estimates $d$, the typical lifespan of this kind of fact. A start-date estimator predicts $t_s$, the time the parametric answer became true. It is trained on the change dates available in SituatedQA and DYNAMICQA. The expected-expiry score is
$$ s = P\left(t_q - t_s > d\right), $$
where $t_q$ is the time of the conversation. The threshold on $s$ is tuned on a development split.
When $s$ is low, the system uses ConvGQR-style expansion unchanged. When $s$ is high, it runs one of two variants, and we compare them. The first, "de-anchored" expansion, masks the specific answer entity in the guess and keeps its relation and type words. The second, "grounded" expansion, retrieves once with the rewrite alone, regenerates the guess from the top passages, and then expands. The gate adds one small classifier call per turn. Grounded expansion adds a second retrieval, but only on gated turns.
Baselines
The naive reference point is the rewrite with no expansion, using ConvGQR's rewriting model alone. This shows whether expansion helps or hurts on temporal turns at all. A second naive reference is raw concatenation of the dialogue history as the query, which Liusie et al. report can preserve keywords that rewriters lose.
The main competitor is ConvGQR with its guessed-answer expansion always on. The proposal is a direct modification of this system, and it is the design most exposed to the failure. Its reported QReCC dense MRR is 42.0 (ConvGQR).
The strongest existing reformulators are AdaCQR and ConvSearch-R1. AdaCQR reports an MRR of 32.8 on TopiOCQA with dense retrieval and 52.4 on QReCC with sparse retrieval (AdaCQR). ConvSearch-R1 reports gains of over 10% in MRR@3 without an absolute figure in the file (ConvSearch-R1). ConvSearch-R1 matters in particular because its reasoning step is the most plausible existing mechanism that could already notice staleness. On our temporal benchmark there is no reported result found on Lacuna for any of these systems.
All systems are run under the same conditions. They use the same BM25 and ANCE indexes over the mixed corpus and the same QReCC training data. Each gets an equal hyperparameter search budget on a shared development split. All use the same base expansion model, so that differences in knowledge cutoff do not confound the comparison.
Two ablations isolate the gate. The first replaces it with a random gate that fires at the same rate. The second uses the gate without the start-date term, which recovers Zhang and Choi's lifespan-only signal. An oracle gate that knows which turns are truly stale provides an upper bound.
Beating the baselines means two things together. On temporal turns, the gated system must lower the stale-hit rate below always-on ConvGQR, with confidence intervals that do not overlap, and match or exceed ConvGQR's MRR there. On the static controls and on the original QReCC, TopiOCQA and CAsT test sets, its MRR must stay within one point of ConvGQR.
Why not just…?
Why not just drop answer expansion entirely? If the harm is real, disabling expansion avoids it. But ConvGQR's gains over human rewrites (42.0 vs 38.4 on QReCC) come largely from expansion, and SituatedQA suggests only about 16.5% of information-seeking questions change over time. Turning expansion off everywhere would sacrifice most turns to protect a minority. The no-expansion baseline measures this trade-off directly. If it matches the gated system on the full test sets, the gate is unnecessary.
Why not just prepend today's date? RealTime QA reports that prepending the date helps open-book models but hurts closed-book ones. The expansion step is closed-book. SituatedQA also shows models handle explicit temporal modifiers poorly, with 23% EM against 57% for humans. We still include date-prompted expansion as a baseline, because it costs nothing to run.
Why not just always ground the guess on a first retrieval pass? This is the real threat. Two-pass grounding resembles classic pseudo-relevance feedback, needs no volatility model, and could match our results. We run it on every turn as a baseline. If it equals the gated system on both temporal and static turns, the honest conclusion is that the gate's value is only efficiency: fewer second retrievals. We would then report the gate's cost savings, not accuracy gains.
The risk also runs the other way. RealTime QA attributes 34% of errors to retrieval failure, so when the first pass returns stale passages, grounding may simply copy the stale answer. The stale-hit rate of the first pass tells us which of these outcomes holds.
Proposed Experiments
The first check tests the premise. On the temporal benchmark, we compare the rewrite without expansion against always-on ConvGQR, split by stale-prone and control turns. The premise holds if expansion raises the stale-hit rate on stale-prone turns and not on controls. The stale-hit rate is the fraction of the top 10 passages that support a superseded answer. If there is no difference, the proposal reduces to a negative result, and we report it as one.
The second check tests the gate. We compare the full gate, the ablations, and the oracle, reporting MRR, NDCG@3 (a ranking-quality score over the top three results), and stale-hit rate under both BM25 and ANCE. The start-date ablation shows whether estimating when a fact began improves on lifespan alone, as the open problem in Zhang and Choi suggests it should.
The third check runs downstream. A fixed reader reads the retrieved passages and we score current-answer EM and outdated-answer rate. This tests whether retrieval-level staleness actually produces confident outdated answers, as RealTime QA and DYNAMICQA would predict.
The fourth check tests the reasoning concern. We evaluate ConvSearch-R1 on stale-prone turns and inspect whether its reasoning traces reproduce the stale entity. This would carry the chain-of-thought failure reported for agent memory over to search queries.
Finally, we rerun every system on the original QReCC, TopiOCQA and CAsT-20 test sets. The goal is to confirm that the gate does not trade away the static-fact performance that made guessed-answer expansion worth using.
References
- ConvGQR: Generative Query Reformulation for Conversational Search
- IterCQR: Iterative Conversational Query Reformulation with Retrieval Guidance
- AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignment
- ConvSearch-R1: Enhancing Query Reformulation for Conversational Search with Reasoning via Reinforcement Learning
- RealTime QA: What's the Answer Right Now?
- SituatedQA: Incorporating Extra-Linguistic Contexts into QA
- DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models
- Mitigating Temporal Misalignment by Discarding Outdated Facts
- Can Agent Memory Systems Track Evolving State?