Automated Evidence Retrieval and Claim Verification Systems

This direction is merged into another direction: fact-checking scalability. Its last version stays readable and citable here.

Version 1 of 125 papers25 conceptsfrozen 2026-10-04

The internet generates information at a velocity that far outpaces human scrutiny. When a viral claim emerges on social media or a knowledge base like Wikidata is edited, professional fact-checkers face a "scalability bottleneck": there is simply too much content to verify manually. To address this, researchers have developed automated systems that don't just guess if a statement is true, but actively "read" the web to find supporting or refuting evidence.

The Fact-Checking Pipeline

Modern automated fact-checking is rarely a single step. Instead, it is typically structured as a multi-stage pipeline designed to narrow down the vastness of the internet into a single verdict. As demonstrated by the UKP-Athene system, this process generally follows three phases:

  1. Document Retrieval: Identifying relevant sources (like Wikipedia articles or news sites) from a massive corpus.
  2. Evidence Selection: Pinpointing specific sentences within those documents that directly address the claim.
  3. Veracity Prediction: Determining if the selected evidence supports, refutes, or provides insufficient information to judge the claim.

By breaking the problem down, systems can use different specialized models for each task. For instance, Amir Soleimani and colleagues showed that using separate BERT models for retrieval and verification can achieve state-of-the-art recall, ensuring that the system doesn't miss critical evidence before making a judgment.

Building Intuition with Attention

A core challenge in this field is "explainability." A black-box label of "False" is less useful than a verdict accompanied by the specific sentence that disproves the claim. To solve this, models like DeClarE, developed by Kashyap Popat and Andrew Yates, use claim-specific attention mechanisms.

Instead of treating all words in a source document equally, the model uses the claim itself as a query to "attend" to the most relevant parts of the evidence. This allows the system to provide human-readable rationales. Similarly, the FAKTA framework integrates stance detection to understand not just if a document mentions a topic, but what position it takes relative to the claim.

Joint Learning and Triage

While pipelined systems are common, they can suffer from "error propagation"—if the retrieval step fails, the verification step has no chance of success. To mitigate this, Sizhen Li and Bo Cheng proposed EMBA, which uses multi-task learning to perform evidence retrieval and claim classification simultaneously. This joint approach allows the model to learn the relationship between evidence and veracity more holistically.

However, even the best automated systems cannot replace humans entirely. This has led to the concept of check-worthiness estimation or "triage." As explored in the CLEF CheckThat! Lab, the goal is to rank claims by their potential impact and falsehood, allowing professional fact-checkers to prioritize their limited time on the most damaging misinformation.

The Shift to Generative Models

The landscape of fact-checking is currently being transformed by Large Language Models (LLMs). Rather than relying on static classifiers, newer systems like OldJoe utilize Retrieval-Augmented Generation (RAG). In this setup, an LLM like Qwen or Llama acts as an investigative agent, breaking down complex claims into smaller questions, searching for answers, and synthesizing a final report.

As noted in the survey by Alphaeus Dmonte and Isabelle Augenstein, this new era brings new challenges, such as "hallucinations" where the model might invent facts. Interestingly, research by Shorouq Zahra suggests that while large models are powerful, specialized Natural Language Inference (NLI) models are often more reliable referees for detecting these internal errors, suggesting a future where small, precise models act as guards for larger, generative ones.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_automated_evidence_retrieval_and_claim_v_5988c74a,
  title = {Automated Evidence Retrieval and Claim Verification Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-evidence-retrieval-and-claim-verification-systems/rdr_5988c74ae9cd48c196d2bbad622b9b75},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_automated_evidence_retrieval_and_claim_v_5988c74a_v1,
  title = {Automated Evidence Retrieval and Claim Verification Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-evidence-retrieval-and-claim-verification-systems/rdr_5988c74ae9cd48c196d2bbad622b9b75/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}