A Systematic Survey of Claim Verification: Corpora, Systems, and Case Studies

Zhaxi Zerong, Chenxi Li, Xinyi Liu, Jinfu Chen, Fei Xia2025 · DOI

This systematic survey by Zerong et al. (University of Washington) provides a comprehensive map of the "Claim Verification" (CV) landscape, analyzing 198 studies published between 2022 and 2025. While fact-checking often involves finding evidence from the open web, CV focuses on the core reasoning task: given a claim and a specific set of reference documents, is the claim supported, refuted, or is there not enough information?

Abstract

The paper synthesizes three years of rapid progress in automated claim verification, moving beyond simple "true/false" labels to explore complex system architectures and multimodal datasets. Through two rigorous case studies, the authors reveal that current "best practices"—like breaking claims into smaller pieces or relying on isolated sentences for evidence—often fail to capture the nuance required for real-world accuracy.

Methodology

The authors conducted a systematic literature review, filtering 316 initial papers down to 198 core studies. They categorized these into two main buckets: Corpus Construction (how "gold standard" datasets are built) and System Development (how models are built, from traditional pipelines to LLM-based agents).

The standout methodological contribution is the inclusion of two in-depth case studies. Instead of just reporting what other papers did, the authors performed their own manual re-annotations of popular datasets to test the limits of "Evidence-Bearing Sentences" (EBS) and "Claim Decomposition."

The Dataset Landscape

The survey reviews 65 newly created CV corpora. While FEVER remains the foundational benchmark, newer datasets like HealthFC (medical), MSVEC (scientific), and WiCE (Wikipedia-based) are pushing the field toward domain-specific verification.

There is also a notable shift toward multimodal evidence; 13 of the new corpora include non-textual evidence like images, charts (ChartCheck), and tables (FACTLENS). However, the paper notes a trend toward using LLMs to generate "artificial" claims to balance datasets, which may create a distribution shift where models fail on real-world nuances.

Critical Insights from Case Studies

The Decomposition Dilemma

A popular strategy in modern CV systems is to break a complex claim into simpler sub-claims. However, the authors' analysis revealed a significant "Decomposition Dilemma." They found that 24% of these decompositions lose "semantic equivalence" to the original claim. If the sub-claims do not perfectly represent the original assertion, the final verdict will be flawed regardless of the model's reasoning capabilities.

Semantic equivalence of claim decompositions.

Linguistic Sources of Error

In Case Study #2, the authors identified six common strategies for claim decomposition and analyzed where they fail. They found that "coordinating conjunctions" (splitting sentences using "and" or "but") are the most frequent source of semantic errors, often leading to problematic sub-claims that misrepresent the original context.

Correct vs. problematic claim decompositions by linguistic category.

The Context Trap

Most CV systems are designed to find a few "Evidence-Bearing Sentences" (EBS) and ignore the rest. The authors found that humans struggle to verify claims using these isolated snippets because they lack context (e.g., pronouns like "it" or "they" referring to something in a previous paragraph). This "Pipeline Fragility" means that if a sentence selector misses one crucial piece of context, the predictor has no chance of success.

Key Results and Limitations

  • Low Human Agreement: Human annotators only agreed on veracity labels about 76% of the time, suggesting that "truth" in complex domains like medicine is often a gradient rather than a binary.
  • LLM Prior Knowledge Conflict: LLMs often "hallucinate" based on internal training data rather than sticking to the provided evidence, especially if the evidence contradicts their pre-trained beliefs.
  • Language Gap: Despite the global nature of misinformation, 50 of the 65 new corpora are English-only.

Future Directions

The authors argue that future research must treat context as a "first-class citizen," including relevant surrounding text alongside evidence-bearing sentences. There is also a pressing need for more multilingual and multimodal datasets, and a shift away from artificial AI-generated claims toward evaluating systems on the messy, "in-the-wild" claims found in social media and political discourse.

Summary updated

BibTeX
@misc{zerong2025systematic,
  author = {Zhaxi Zerong and Chenxi Li and Xinyi Liu and Jinfu Chen and Fei Xia},
  title = {{A Systematic Survey of Claim Verification: Corpora, Systems, and Case Studies}},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.1170},
}
Figure