Automated fact-checkers retrieve web evidence for a claim and then ask a language model to judge it. Claims are short and colloquial, while useful evidence is long and formal, so the best current systems first have a large language model write an imagined fact-check article and then search with that text. This step closes the vocabulary gap between claim and evidence. It also adds a hidden problem. The model writes in the vocabulary of its own training period, not in the vocabulary of the time the claim was made. The expanded query therefore pulls in documents that use the right words about the wrong moment. These include retrospective articles written after the verdict was known, and stale reports from before the facts changed. The reader model then receives evidence that matches the claim on topic but not on time, which is the kind of distractor that language models are known to over-trust.
We propose to measure this failure and repair it where it starts. The repair is to draw the expansion vocabulary from documents dated near the claim, instead of from the model's memory alone. Expansion terms that first appear in the corpus after the claim date get filtered out. We then test whether this reduces verdict errors caused by topically relevant but temporally displaced evidence. Our working claim is that much of what fact-checking papers call "retrieval noise" is temporal noise, and that LLM query expansion is one of its main sources.
Core Claim
Vocabulary mismatch in fact-checking is partly a temporal phenomenon: the words used to describe an event drift after it happens. A hypothetical document written from a model's present-day knowledge therefore leans toward evidence from the model's era rather than the claim's era. We make three predictions. First, a measurable share of the terms in LLM-generated expansions are anachronistic, meaning they are first attested in the dated corpus after the claim date. Second, anachronism rate predicts both temporal leakage and verdict errors. Third, an expansion grounded in a time slice of the corpus, with anachronistic terms removed, keeps most of the recall gain of expansion while cutting temporally displaced evidence. A hard date filter applied after retrieval cannot do this alone, because it removes future documents without fixing how pre-claim documents are ranked or the drift in the query itself.
Where This Sits in Prior Work
The closest system supplies the method we modify. HerO introduced HyDE-FC: Llama-3.1-70b writes a hypothetical fact-checking article that is used as the query for BM25 plus dense reranking on AVeriTeC. The approach placed second in the shared task, and its authors name noisy search snippets as the main bottleneck. HyDE-FC applies no temporal bound to generation; our proposal changes exactly that step. InteR alternates LLM expansion and retrieval rounds, and reports that more than two rounds pile up noise. We reuse its feedback loop but restrict the retrieval side to a time window, so the feedback anchors vocabulary to the claim's era rather than only to topic. RaFe trains small query rewriters from reranker scores. Its rewrites are atemporal, and it suffers from reranker bias, which a temporal signal could worsen if the reranker prefers recent, well-written retrospectives.
On the temporal side, Implicit Temporal Reasoning for Evidence-Based Fact-Checking bins the gap between claim date and evidence date into embeddings for the classifier, raising BiLSTM macro-F1 on MultiFC from 32.39% to 47.43%. This motivates our work: time works as a relevance filter. The same authors state that temporal reasoning should move into retrieval, and that is the step we take. AVeriTeC splits claims by date to prevent leakage and documents that systems verify old claims with later articles that already contain the verdict. It supplies our setting and our leakage concern. The MultiFC study shows that web evidence for a claim shifts over time. Community-QA work (Mihaylova et al.) shows that answers can be true when written and false later. Both motivate treating evidence that is old but on-topic as a distinct kind of distractor.
On noise, ATTUN trains attention to ignore distractors and stays robust under 500% random distractor injection. Its distractors are topically irrelevant, so it has no notion of a relevant-but-wrong-era document. DAS shows that zero-shot readers assign confidence as high as 0.98 to wrong answers drawn from distractor passages. It recovers accuracy through unanswerability prompting and query-likelihood rescoring. Both readers become baselines on our temporal stress test. Ev2R scores retrieved evidence by atomic facts and correlates with human judgments far better than token-overlap metrics. We use it as an evaluation metric, noting that its atomic facts carry no timestamps.
Research Program
The work has three parts: a diagnostic, a retrieval method, and a stress test for readers.
The diagnostic needs a term-first-seen index. For each content term and noun phrase in the dated document collection, we record the earliest publication date at which it appears. For a claim with date $t_c$ and an expansion text $E$, the anachronism rate is
$$A(E, t_c) = \frac{|\{w \in E : \text{first}(w) > t_c\}|}{|E|},$$
where $\text{first}(w)$ is the earliest corpus date of term $w$, and the sets range over content terms of $E$. We compute $A$ for HyDE-FC expansions on AVeriTeC dev and test, and on MultiFC claims. MultiFC carries the publication-date metadata that Allein et al. already used. For AVeriTeC, evidence dates must be recovered from page metadata or extracted text, and we will report coverage honestly, because the file notes that snippets often lose timestamps. We then measure whether $A$ correlates with the fraction of retrieved evidence that post-dates the claim, and with veracity errors.
The method, time-sliced feedback expansion, has three stages. First, retrieve with the claim alone, restricted to documents dated within a window $[t_c - \Delta, t_c]$. Second, have the same LLM HerO uses write the hypothetical fact-check, conditioned on the claim, its date, and the top in-window documents. This is one InteR-style round, capped at two. Third, remove terms with $\text{first}(w) > t_c$ and run the hybrid retriever. Retrieved evidence is passed to the reader together with its dates. The window $\Delta$ is tuned on the AVeriTeC dev split only.
The stress test builds, for each claim, two matched distractor sets of equal size and equal lexical overlap with the claim. One is topical-irrelevant, in ATTUN's style. The other is topical-but-temporally-displaced: documents about the same entities dated well before or after $t_c$. We report how often verdicts flip and how confident the reader is when they do, under each kind of distractor.
Baselines
The naive reference is claim-only BM25 plus dense reranking feeding the same reader. This shows how much any expansion helps at all.
The strongest existing competitor is HerO's HyDE-FC pipeline, run as published: Llama-3.1-70b expansion, BM25 plus SFR-embedding-2, and the same fine-tuned CoT verdict model. HerO reports an AVeriTeC answer-quality "A score" of 0.2801 against a shared-task baseline of 0.187, an overall shared-task score of 0.57, and 75.2% development veracity accuracy. These numbers make it the bar on this benchmark. We rerun it ourselves on identical document collections and give it the same number of LLM generation calls our method uses, by sampling extra hypothetical documents. This way any gain cannot come from extra compute.
The second strong baseline is HyDE-FC followed by a hard filter that removes evidence dated after $t_c$. This is the obvious leakage fix.
The third is HyDE-FC with the reranker augmented by Allein et al.'s date-distance buckets as a feature. This tests whether temporal information at reranking time suffices.
On MultiFC, the reference points are the reported macro-F1 of 49.2% for the best MultiFC model and 47.43% for Allein et al.'s temporally grounded BiLSTM.
For readers on the stress test, we compare a vanilla CoT reader, DAS-style unanswerability prompting with query-likelihood rescoring, and ATTUN. The file has no reported AVeriTeC-specific ATTUN number, so we report our own runs.
Three ablations isolate the key ingredient: date conditioning in the prompt only, time-sliced feedback without the anachronism filter, and the anachronism filter applied to plain HyDE-FC. The method counts as beating HyDE-FC only if three conditions hold together. It must match or exceed HyDE-FC's A score and Ev2R evidence score, lower the fraction of post-claim evidence, and lower the temporal-distractor flip rate. It must also beat the date-filter baseline on at least the Ev2R score and flip rate.
Why not just…?
Why not just filter out documents published after the claim date? This is the most serious threat, and we include it as a baseline rather than dismissing it. A filter removes leakage, but the query still carries the drifted vocabulary. After filtering, anachronistic terms match nothing, so recall should fall. The filter also does nothing about stale pre-claim documents, which is the perishable-truth case documented by Mihaylova et al.. If filter plus HyDE-FC matches our method on Ev2R score and flip rate, then the drift hypothesis is wrong. We would report that and keep the stress test and the anachronism diagnostic as the contribution.
Why not just tell the LLM the claim date when it writes the hypothetical article? This costs nothing and may remove much of the anachronism. Our date-prompt-only ablation tests it directly. We expect partial success: a model cannot reliably know which words did not yet exist at a given date, and corpus feedback supplies this knowledge empirically. The diagnostic will show how much of $A$ the prompt alone removes.
Why not just make the reader robust, with ATTUN or DAS, and leave retrieval alone? Robust readers trained on topical distractors may not reject on-topic, wrong-era evidence, because nothing in their training signal marks time. The matched stress test answers this directly. If ATTUN's flip rate on temporal distractors equals its rate on topical ones, then reader-side robustness is enough and retrieval changes matter less.
Why not just add a date-proximity feature to the reranker? Allein et al.'s gains suggest this will help. It acts only on what was already retrieved, though, and their own result implies that upstream misses cannot be recovered downstream. We test this as a baseline, and also in combination with our method, to check whether the two effects add up.
Proposed Experiments and Decisive Checks
On AVeriTeC we use the official chronological split, tune on dev, and report test results. Metrics are A score and overall score, Ev2R evidence scores, veracity accuracy and macro-F1, fraction of evidence post-dating the claim, and mean anachronism rate. On MultiFC we report macro- and micro-F1 with the same retrieval variants over its snippet pool, which tests whether the effect holds beyond one benchmark. On both, we run the matched topical and temporal distractor stress test for every reader.
One confound needs care. HyDE-FC's generator was trained long after the 2021 test claims, so its hypothetical articles may already contain the verdict. Part of HyDE-FC's measured score may therefore be leakage, and an honest method could score lower on raw accuracy. We address this by also reporting accuracy restricted to claims whose retrieved evidence is entirely pre-claim, and by checking whether HyDE-FC's advantage concentrates on claims with high anachronism.
The project succeeds or fails on three results. First, the HyDE-FC anachronism rate must be clearly non-zero and predictive of leakage and errors. If it is near zero, the premise fails and we stop. Second, time-sliced feedback expansion must beat the hard-filter and date-prompt baselines on Ev2R score and temporal flip rate at matched LLM budget. Third, its gains should concentrate on claims about fast-changing situations, such as the 2020–2021 claims spanning the dev and test splits, rather than spreading evenly across all claims.
References
- HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claims
- Synergistic Interplay between Search and Large Language Models for Information Retrieval
- RaFe: Ranking Feedback Improves Query Rewriting for RAG
- Implicit Temporal Reasoning for Evidence-Based Fact-Checking
- AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web
- MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims
- SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums
- Refining Attention for Explainable and Noise-Robust Fact-Checking with Transformers
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking