BiDeV: Bilateral Defusing Verification for Complex Claim Fact-Checking

Abstract

Complex claim fact-checking performs a crucial role in disinformation detection. However, existing fact-checking methods struggle with claim vagueness, specifically in effectively handling latent information and complex relations within claims. Moreover, evidence redundancy, where non-essential information complicates the verification process, remains a significant issue. To tackle these limitations, we propose Bilateral Defusing Verification (BiDeV), a novel fact-checking working-flow framework integrating multiple role-played LLMs to mimic the human-expert fact-checking process. BiDeV consists of two main modules: Vagueness Defusing identifies latent information and resolves complex relations to simplify the claim, and Redundancy Defusing eliminates redundant content to enhance the evidence quality. Extensive experimental results on two widely used challenging fact-checking benchmarks (Hover and Feverous-s) demonstrate that our BiDeV can achieve the best performance under both gold and open settings. This highlights the effectiveness of BiDeV in handling complex claims and ensuring precise fact-checking.

Introduction

Automated fact-checking has transitioned from simple sentence-level verification to the analysis of "complex claims" that necessitate multi-step reasoning across various sources. Despite advancements, two primary obstacles hinder current systems: claim vagueness and evidence redundancy.

Real-world claims are frequently underspecified, containing latent information—such as unresolved entities or undetermined attributes—and complex relations, including referential dependencies and comparative logic. For instance, a claim mentioning "the writer of the novel Horizon" without a name requires identifying the entity before verification can proceed. Furthermore, retrieving evidence from open-web sources often introduces "noisy" or redundant data that distracts Large Language Models (LLMs), leading to reasoning failures.

Expand

Source, p. 1

Methodology: The BiDeV Framework

To address these challenges, we introduce BiDeV (Bilateral Defusing Verification), a multi-agent framework that assigns specialized roles to LLMs to simulate a human expert's reasoning process. The framework is composed of two core modules: Vagueness Defusing and Redundancy Defusing.

Vagueness Defusing (VD)

The VD module simplifies complex claims through a two-stage process:

1. Perceive-then-Rewrite: A Perceptor identifies missing or ambiguous information. A Querier then fetches the necessary background knowledge, which a Rewriter integrates into the claim. This loop typically iterates three times to ensure the claim is explicit.
2. Decompose-then-Check: Once clarified, a Decomposer breaks the claim into atomic, verifiable sub-claims. A Checker then verifies each sub-claim against the filtered evidence.

Redundancy Defusing (RD)

To mitigate the impact of noisy evidence, the RD module employs a Filter agent. This agent evaluates retrieved evidence paragraphs and discards irrelevant content, ensuring that subsequent reasoning agents only process high-quality, pertinent information.

Expand

Source, p. 3

Experimental Setup

BiDeV was evaluated on two prominent benchmarks for multi-step reasoning:

• Hover: A dataset designed for many-hop fact extraction and claim verification, featuring 2-hop, 3-hop, and 4-hop claims.
• Feverous-s: A subset of the FEVEROUS dataset focusing on complex claims that require evidence from both unstructured text and structured tables.

The framework is model-agnostic; experiments primarily utilized gpt-3.5-turbo for agent roles, with additional testing on the FLAN-T5 family to assess scalability.

Results and Discussion

BiDeV achieved state-of-the-art performance, outperforming 11 competitive baselines, including Hiss, ProgramFC, and FactcheckGPT.

Key Performance Metrics

• Overall Accuracy: BiDeV improved Macro-F1 scores by an average of 3.88% across both "gold" (provided evidence) and "open" (retrieved evidence) settings.
• Complexity Scaling: The performance advantage of BiDeV became more pronounced as claim complexity increased. On the most challenging 4-hop claims in the Hover dataset, BiDeV achieved a 17.72% improvement over fine-tuned baselines.
• Robustness: Structured workflows allowed smaller models (e.g., FLAN-T5-base) to outperform significantly larger 11B parameter models, demonstrating that the process of "defusing" is as critical as raw model parameters.

Ablation Insights

Ablation studies highlighted the Perceptor as the most critical role; identifying "what the model doesn't know" is foundational to successful verification. Additionally, the Filter module proved essential in open-setting scenarios, where evidence redundancy otherwise significantly degraded performance.

Conclusion

BiDeV demonstrates that a structured, multi-agent approach to "defusing" claim vagueness and evidence noise significantly enhances fact-checking accuracy for complex claims. While the iterative nature of the framework introduces higher latency and cost compared to end-to-end models, the gains in precision and robustness are substantial. Future work will explore specialized roles for multi-modal evidence and further optimization of agent interactions.

@inproceedings{liu2025bidev,
  author = {Yuxuan Liu and Hongda Sun and Wenya Guo and Xinyan Xiao and Cunli Mao and Zhengtao Yu and Rui Yan},
  title = {{BiDeV: Bilateral Defusing Verification for Complex Claim Fact-Checking}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=vTYfK46KMM},
}