Fact Checking Beyond Training Set

Payam Karisani, Heng JiNAACL 2024 · DOI

Abstract

Automated fact-checking systems typically rely on a two-stage "retriever-reader" architecture. While effective within a specific training domain, these systems often fail when applied to new, unseen topics—a phenomenon known as domain shift. This paper presents a novel framework designed to enhance the robustness of fact-checking pipelines across diverse domains. By employing adversarial training for domain-invariant evidence retrieval and data augmentation to ensure reader stability, the proposed model achieves significant performance gains over state-of-the-art baselines and general-purpose large language models (LLMs).

Introduction

The proliferation of misinformation necessitates scalable, automated fact-checking solutions. Most modern systems utilize a pipeline where a retriever identifies relevant evidence from a corpus (e.g., Wikipedia) and a reader (a classifier) evaluates the claim against this evidence to produce a verdict of "Supported" or "Refuted." However, these systems are often brittle. A model trained on political discourse may struggle with the specialized terminology of healthcare or the nuances of celebrity news. Furthermore, while LLMs like GPT-4 show promise, they are limited by knowledge cutoffs and sensitivities to information ordering.

Methodology

The authors propose a domain-agnostic pipeline designed to mitigate the impact of domain shift through three key innovations.

The standard two-stage fact-checking (FC) pipeline architecture.

1. Adversarial Retriever Training

To ensure the retriever generalizes to new domains without additional labeled data, an adversarial training regime is implemented. A discriminator is trained to distinguish between text snippets from the source and target domains. Simultaneously, the claim and document encoders are optimized to "fool" this discriminator. This process encourages the encoders to extract domain-invariant features, preventing the model from over-relying on source-specific keywords that may not appear in the target domain.

2. Order-Insensitive Reader

The "reader" component often suffers from a "reversal curse," where the logical symmetry between a claim and its evidence is not fully captured. To address this, the authors augment the training data by permuting the order of claims and evidence. Additionally, a distance-based discrepancy reduction loss is introduced to align the internal representations of source and target data, ensuring the classifier remains stable regardless of the topic.

3. Weighted Inference

During the final verdict stage, the system does not treat all retrieved documents equally. Instead, a ranking-based weighting system is applied, where top-ranked documents from the retriever exert a greater influence on the final classification.

Experimental Setup

To evaluate domain adaptation, the authors re-purposed two major datasets:

  • MultiFC: Categorized into five domains (Arts, Business, Misc, Politics, Sensitive) using the Google Content Classifier.
  • Snopes: Categorized into General and News.

By alternating between these domains as "source" (labeled) and "target" (unlabeled), the researchers established eight distinct domain-shift scenarios.

Key Results and Analysis

The proposed model consistently outperformed state-of-the-art domain adaptation models, such as GPL and Promptagator.

F1 performance of the end-to-end pipeline across different source and target domains.
  • Retrieval Generalization: The adversarial retriever achieved an average NDCG@10 of 0.784 on MultiFC, outperforming standard BM25 (0.711) and other neural retrievers. This confirms that the adversarial approach effectively stabilizes search across domains.
  • End-to-End Accuracy: The complete pipeline reached an average Macro F1 of 0.618 on MultiFC and 0.437 on Snopes.
  • Comparison with LLMs: In direct comparisons, the specialized pipeline significantly outperformed GPT-3 (using in-context learning). For instance, in the Misc-to-Arts (M→A) scenario, the model achieved an F1 of 0.595 compared to GPT-3's 0.456. This highlights the continued relevance of specialized, retrieval-augmented systems over general-purpose models, especially when dealing with real-time information or specific domain constraints.

Conclusion

This research underscores the critical role of retrieval in the fact-checking pipeline. By making both the retriever and the reader domain-agnostic through adversarial training and symmetry-aware data augmentation, the authors provide a robust framework for automated verification that extends beyond the limitations of its training set. Future work will look to incorporate multi-modal evidence and address label shift across domains.

Summary updated

BibTeX
@inproceedings{karisani2024fact,
  author = {Payam Karisani and Heng Ji},
  title = {{Fact Checking Beyond Training Set}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.124},
}
Figure