Early web question-answering systems got good results without deep parsing. They relied on redundancy: on a large enough collection, a fact is stated many times, and some of those statements are phrased simply, so counting simple matches beats parsing hard sentences. That bet assumes the counted answer does not change over time, and it fails when the answer does. When the AskMSR system took answers found on the live web and checked them against an older news collection, contemporary officeholders outvoted the historically correct ones. That mismatch cost about 0.09 in mean reciprocal rank (Data-Intensive Question Answering). Modern language-model systems hit the mirror-image problem, knowledge obsolescence. A model frozen at a training cutoff states outdated answers confidently. Handing it fresh documents does not fully fix this: retrieval-based readers are 15 points less accurate on updated facts than on stable ones, even with current Wikipedia (SituatedQA).
This proposal stops treating redundancy as one pile of counts and splits it by publication date. For a question like "who coaches team X," we count, in each time window, how many independent dated sources link the question entity to each candidate answer. A change of fact then shows up as a crossover: the old answer's share falls and a new answer's share rises past it. From these curves we get three things a single retrieved passage cannot give. The first is which answer holds at the query time. The second is roughly when it last changed, which yields measured lifespans for each relation type. The third is a principled reason to abstain when no answer has enough recent support. The core claim is that time-sliced, burst-corrected mention counts outperform both time-blind voting and "read the newest passage" on recently changed facts, specifically in calibration and in knowing when to abstain.
Core Claim and What Is New
The work closest to this splits into two camps that treat redundancy in opposite ways. Classical extraction uses frequency as evidence of truth. Ravichandran and Hovy's bootstrapped surface patterns score precision as answer-matching frequency over pattern frequency (Learning surface text patterns). KnowItAll validates extractions with pointwise mutual information computed from search-engine hit counts, which raises Country-class precision from 0.35 to 0.79 (KnowItAll). AskMSR tiles and votes over snippet n-grams. All three pool counts across time, and that pooling is the source of the projection drop.
Temporal work, in contrast, treats redundancy as noise. TR-Rules collapses repeated events inside a time window into one instance so that bursts do not inflate rule confidence on temporal knowledge graphs (TR-Rules). T-GRAG splits curated annual reports into per-year subgraphs to keep information from other years from diluting the context (T-GRAG). Both operate on structured or explicitly year-bounded data.
Zhang and Choi predict how long a fact typically lasts from the statement's wording alone, using a BERT cloze prompt, and discount confidence once the gap between query time and evidence time exceeds that predicted lifespan (Mitigating Temporal Misalignment). Their model never looks at corpus evidence of whether the fact has actually changed.
We reconcile the two views of redundancy. Within a short window, repeated copies of one news event are squashed to independent sources, following TR-Rules' window confidence but applied to unstructured text. Across windows, the squashed counts are used constructively, as classical extraction intended, to detect state changes. To our knowledge no paper in the file computes per-window answer shares from dated text, or uses their crossovers to answer questions, estimate fact lifespans, or decide when to abstain.
The quantity at the center is the burst-corrected share of candidate answer $a$ in time window $w$:
$$s_a(w) = \frac{n_a(w)}{\sum_b n_b(w)},$$
Here $n_a(w)$ counts distinct (publisher, day) pairs in window $w$ whose text links the question entity to $a$ through an accepted pattern or extraction, and the sum runs over all candidates $b$. Counting publisher-days rather than raw mentions is the squashing step: a syndicated story repeated across 300 pages counts once per outlet per day. The system's answer at query time $t_q$ is the candidate with the highest share in the most recent window whose total support $\sum_b n_b(w)$ exceeds a threshold $\tau$. If no recent window clears $\tau$, or no candidate's share clears a margin, the system flags the question as stale or unanswerable. This plays the same role as KnowItAll's signal-to-noise stopping rule.
Research Program
The system is built in three layers, and the experiments keep them separable so that the contribution of temporal aggregation can be isolated.
The first layer gathers candidates. For each question we retrieve a large set of dated documents rather than a top-10 list. The sources are dated news, such as the article streams behind RealTime QA, and successive Wikipedia dumps for slower-changing facts. A small language model then extracts, from each passage, the candidate it supports for the question relation. Where a relation has stable phrasing, such as "head coach of," bootstrapped surface patterns in the style of Ravichandran and Hovy are used instead. Extraction is deliberately cheap and identical across all compared methods.
The second layer aggregates the candidates. It squashes them to publisher-days, computes the shares $s_a(w)$ over sliding windows, and runs a simple change-point test that finds the window where the leading candidate switches with sustained support. Window width is tuned per relation on the training split. A sports roster and a head of state change at very different speeds.
The third layer turns the detected crossovers into empirical lifespan distributions for each relation type. These replace the cloze duration predictor in Zhang and Choi's confidence-discounting scheme. The result is a corpus-measured answer to "has this probably expired" rather than a guess from the statement's wording.
The open questions are three. Does burst correction matter, or do raw counts work just as well? How large is the detection lag, meaning how many days after a real change the new answer overtakes the old one? Does the lag hurt more than the noise robustness helps? The lag is the method's inherent weakness. A reader that sees one fresh article announcing a change can be right on day one, while consensus needs time to build. We expect, and will measure, a crossover point in time-since-change where consensus begins to win.
Baselines
The naive reference point is the closed-book language model with no retrieval. On RealTime QA, GPT-3 closed-book scores 44.9% multiple-choice accuracy (RealTime QA). We rerun this with the same open-weight reader used in every other condition, so the number is ours rather than borrowed.
The strongest existing open-book competitor is search-augmented reading as in RealTime QA, where the model reads top-10 news results. GPT-3 with Google search reaches 66.5% multiple-choice accuracy and 34.6 exact match on open-ended generation. In our version, the same reader receives the top-10 dated passages from our document pool, with dates shown, so retrieval coverage is matched. Beating it means higher exact match on the subset of questions whose answer changed within the evaluation period, at equal or better accuracy on stable questions.
A stronger and fairer variant of the same idea is recency-sorted reading, in which the reader sees the newest passages first, or only passages after a cutoff. This is the baseline we most expect to compete. It is tuned with the same validation budget as our window and threshold parameters.
For calibration and abstention on SituatedQA, the strongest prior method is Zhang and Choi's duration-based discounting. It reports a 50–60% reduction in expected calibration error and an adaptive system reaching 20.5 exact match while retrieving for 45% of queries. The best finetuned retrieval model in SituatedQA reaches 23% exact match on temporal questions. We reimplement duration discounting on the same reader and swap only the lifespan source, cloze versus corpus-measured. Beating it means lower calibration error at the same retrieval rate, or the same error with fewer retrievals.
For abstention on time-scoped questions, the reference is the GRPO-trained Qwen2.5-1.5B from When Silence Is Golden. It reports 43.41% exact match and 37.2% abstention F1 on TimeQA-Easy, and 35.75% and 33.5% on TimeQA-Hard. Because TimeQA documents are not publication-dated streams, this comparison is secondary. We use it only where we can attach dated sources.
The ablations isolate the key ingredient. Time-blind voting uses the same extracted candidates but pools all windows, which is AskMSR-style. Time-sliced voting is run without burst squashing, counting raw mentions. Squashing is run without slicing. All ablations use the same candidates, documents, and tuning budget.
Why not just…?
Why not just give the reader the newest passages? This is the real threat, and we treat it as the main comparison rather than a straw man. Its weaknesses are concrete. A single new article often mentions the old answer, as in "former coach," or reports a rumor. RealTime QA attributes 34% of errors to retrieval failure, after which models fall back on stale knowledge rather than abstaining. A newest-passage reader also gives no signal of how settled the answer is. If recency sorting matches our accuracy across all time-since-change buckets, we will drop the accuracy claim. The contribution then narrows to calibration, abstention and lifespan measurement, which recency sorting does not provide. The bucketed analysis in the experiments decides this.
Why not just let the language model reason over all dated passages itself? SituatedQA shows models are fragile under temporal modifiers, and the 15-point gap on updated facts persists even with fresh corpora. The model can read dates but does not reliably weigh many conflicting dated claims. A long-context condition in which the reader receives the same full pool of dated passages that our counter sees tests this directly. If that condition matches us, explicit counting adds nothing.
Why not just use Zhang and Choi's duration predictor, which already handles expiry? It predicts a typical lifespan from wording, and only 65% of its predictions fall within a three-year window of the truth. It cannot know that a particular fact changed last month. Our lifespan swap keeps the rest of their pipeline fixed, so any calibration difference comes from corpus-measured versus wording-predicted duration.
Why not just store facts in a temporal knowledge graph? Building that graph requires the very extraction-and-dating step we study. T-GRAG works on curated annual reports, and When Silence Is Golden found structured knowledge-graph cues gave limited benefit over text. A graph fed by our crossover detections is a reasonable downstream use, not a competitor.
Proposed Experiments
The primary testbed is RealTime QA. Its weekly questions come with publication-dated news, and its open-ended exact match exposes reversion to stale answers. We split questions by time elapsed since the gold answer last changed, using annotation where available and, as a check, the change date our method detects. We report exact match per bucket for every baseline and ablation.
The decisive accuracy check is this. In buckets where the change happened more than a few weeks earlier, time-sliced consensus should beat both time-blind voting, which should still pick the old answer as AskMSR did, and newest-passage reading. In the most recent bucket, we expect to lose to newest-passage reading, and we report the lag in days. A hybrid that falls back to the newest passage whenever the recent-window share is uncertain is evaluated as a planned third arm, not an afterthought.
The second testbed is the temporally dependent SituatedQA subset used by Zhang and Choi, with N=322. We run it with successive Wikipedia dumps as dated snapshots and measure expected calibration error, exact match, and retrieval rate under the lifespan swap.
The third check validates lifespans directly. For relations with known change dates, we compare corpus-measured durations against gold, using the same three-year-window criterion on which the cloze model scores 65%.
The final check targets abstention. On RealTime QA's none-of-the-above items, and on TimeQA questions where dated sources can be attached, we compare abstention F1 from the support threshold $\tau$ against the GRPO-trained abstainer.
The idea fails cleanly if either of two things happens. Burst squashing may show no gain over raw counts, in which case the reconciliation with TR-Rules is empty. Or newest-passage reading may match us in calibration as well as accuracy, in which case temporal redundancy adds nothing beyond recency.
References
- Data-Intensive Question Answering
- SituatedQA: Incorporating Extra-Linguistic Contexts into QA
- Learning surface text patterns for a Question Answering System
- Web-scale information extraction in knowitall: (preliminary results)
- TR-Rules: Rule-based Model for Link Forecasting on Temporal Knowledge Graph Considering Temporal Redundancy
- T-GRAG: A Dynamic GraphRAG Framework for Resolving Temporal Conflicts and Redundancy in Knowledge Retrieval
- Mitigating Temporal Misalignment by Discarding Outdated Facts
- RealTime QA: What's the Answer Right Now?
- When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?