Identifying Unverifiable Claims in Fact Verification

47 concepts36 papers1 proposal

In the idealized version of automated fact-checking, a system receives a claim, searches a corpus, and finds a definitive document that either supports or refutes it. However, real-world information ecosystems are rarely so cooperative. Many claims are "unverifiable"—not because they are false, but because the necessary evidence simply does not exist in the available data.

Distinguishing between a claim that is explicitly refuted and one for which there is Not Enough Info (NEI) is one of the most significant hurdles in the field. As Tuhin Chakrabarty et al. note, this boundary is often ambiguous, leading to systems that struggle to categorize the absence of evidence correctly.

The Asymmetry of Evidence

The core difficulty lies in an epistemological asymmetry: proving a claim is supported or refuted requires finding a specific "needle" in the haystack. Proving a claim is unverifiable requires the system to have high confidence that the needle is not in the haystack at all. Farhad Nooralahzadeh et al. argue that this task is inherently more difficult because it demands a global assessment of the entire corpus rather than a local match.

When systems fail to find evidence, they often default to a "refuted" verdict. Sizhen Li et al. observe that automated models frequently misclassify missing information as a direct contradiction. This confusion stems from a failure to distinguish between a "retrieval failure" (the system missed the evidence) and a "genuine evidence gap" (the evidence does not exist), a distinction highlighted by Chien-Sheng Wu et al..

Data Voids and Infeasible Claims

The problem is exacerbated by the nature of modern misinformation. Max Glockner et al. point out that novel misinformation often exists in "data voids"—topics where no credible counter-evidence has yet been produced. In these scenarios, traditional document-matching models fail because they rely on the existence of explicit contradictory documents that haven't been written yet.

Furthermore, many claims are fundamentally "infeasible" to verify due to being contextually underspecified. Camille Thibault et al. found that a significant portion of claims in existing benchmarks are impossible even for human experts to verify. When models are forced to provide a binary "True" or "False" label for these claims, they are essentially being measured on their ability to hallucinate or guess rather than their reasoning capabilities.

Moving Beyond Binary Verdicts

To address these gaps, researchers are moving toward more nuanced classification strategies and reasoning architectures:

  • Ternary Classification: Instead of binary labels, systems like those proposed by Michael Sejr Schlichtkrull et al. explicitly incorporate a third category for insufficient evidence to balance accuracy with honesty.
  • Confidential Reasoning: New architectures like the CO-GAT model by Yuqing Lan et al. attempt to prevent irrelevant or "neutral" information from polluting the reasoning process, helping the model recognize when the retrieved data is insufficient for a verdict.
  • Explaining the Absence: Transparency is also a priority. Neema Kotonya argues that for a system to be truly explainable, it must be able to articulate exactly what evidence is missing, rather than just pointing to what it did find.
  • Iterative Inquiry: Rather than a "one-shot" retrieval, systems like the one developed by Juraj Vladika et al. use Large Language Models to generate follow-up questions, treating fact-checking as an iterative search for sufficiency.

By treating the absence of evidence as a first-class signal rather than a failure of the algorithm, these approaches aim to make fact-checking systems more robust against the uncertainties of real-world information.

Key Foundational Work

SIRIUS-LTG: An Entity Linking Approach to Fact Extraction and Verification (2018) — contributed 3 concept elements to this direction.
An End-to-End Multi-task Learning Model for Fact Checking (2018) — contributed 1 concept elements to this direction.
Uni-DUE Student Team: Tackling fact checking through decomposable attention neural network (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_insufficient_evidence_detection_f6859018,
  title = {insufficient evidence detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/insufficient-evidence-detection/rdr_f6859018dff8401ba6575a2715792882},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_insufficient_evidence_detection_f6859018_v1,
  title = {insufficient evidence detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/insufficient-evidence-detection/rdr_f6859018dff8401ba6575a2715792882/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake