Scaling Truth: The Confidence Paradox in AI Fact-Checking

Ihsan Ayyub Qazi, Zohaib Khan, Abdullah Ghani, Agha Ali Raza, Zafar Ayyub Qazi, Wassay Sajjad, Ayesha Ali, Asher Javaid, Muhammad Abdullah Sohail, Abdul Hameed AzeemiarXiv 2025 · DOI · arXiv

TL;DR

This paper investigates the performance of Large Language Models (LLMs) in fact-checking, revealing a "confidence paradox": smaller, more accessible models are highly confident but less accurate, while larger, more accurate models are more cautious. This has significant implications for information integrity, potentially exacerbating existing inequalities in access to reliable fact-checking.

What Problem Does This Paper Address?

The increasing integration of LLMs into everyday life, including their use by fact-checking organizations, raises critical questions about their reliability and impact on information integrity. While LLMs can process vast amounts of information, they are prone to "hallucinations" (generating false content) and users often overestimate their accuracy. Furthermore, equitable access to advanced LLMs is a challenge, with smaller, less reliable models being more accessible. This disparity is compounded by performance gaps in multilingual contexts and for claims originating from the Global South, potentially widening existing information inequalities. The paper addresses the need for a systematic evaluation of LLMs in realistic fact-checking scenarios, considering various model sizes, architectures, and prompting strategies across diverse linguistic and geographic contexts.

Methods

The researchers developed a novel methodology to evaluate LLMs in fact-checking that mirrors real-world user interactions. They used a dataset of 5,000 real-world claims, fact-checked by 174 organizations across 47 languages, with ground truth derived from human annotations. Nine established LLMs were evaluated using four prompting strategies: three mimicking typical user interactions (direct truth/falsity inquiry, binary choice) and one using a specialized system prompt designed to adopt a professional fact-checker persona. To ensure robustness, they also tested models on claims published after their knowledge cutoffs. Key metrics included Selective Accuracy (accuracy on definitive judgments), Abstention-Friendly Accuracy (correctness or appropriate abstention), and Certainty Rate (decisiveness).

Datasets

The study utilized a dataset of 5,000 fact-checked claims, compiled from Google Fact Check Explorer. These claims were originally verified by 250 fact-checking organizations worldwide and covered 47 languages. The dataset was carefully curated to include only claims with unambiguous true/false ratings across multiple fact-checking organizations. For temporal analysis, a post-training dataset of 2,112 claims published after the knowledge cutoff of the earliest model was also created.

Key Insights and Ideas

  • The Confidence Paradox: A central finding is the "Dunning-Kruger effect" in LLMs, where smaller, more accessible models exhibit high confidence despite lower accuracy, while larger, more accurate models are more cautious and less confident. This is a critical trade-off for fact-checking systems.
  • Prompting Matters: While user-level prompts are more realistic, a specialized "fact-checker" system prompt significantly improves model performance, especially for larger models. However, this comes at the cost of reduced abstention-friendly accuracy for smaller models.
  • Multilingual and Regional Disparities: LLMs show significant performance drops for non-English languages and claims from the Global South, threatening to widen existing information inequalities. Larger models, while more accurate, exhibit greater caution in these contexts, potentially hindering operational efficiency.
  • Cost vs. Performance: There's a clear trade-off between model performance and computational cost, with cutting-edge models being prohibitively expensive for many organizations.

Key Results

  • Selective Accuracy: Larger models like GPT-4 and GPT-40 achieved high selective accuracy (up to 89%), while smaller models like Llama-7B hovered around 60%.
  • Certainty Rate: Smaller models (Llama-7B, Mistral-7B) showed high certainty rates (up to 88%), while larger models (GPT-40) exhibited lower certainty rates (below 40%) on complex claims.
  • Multilingual Performance: Selective accuracy decreased for Portuguese and Hindi claims compared to English claims (up to 4.3%), while abstention-friendly accuracy generally increased for smaller models on non-English claims.
  • Regional Disparities: Most models performed worse on claims relevant to the Global South, with selective accuracy declining notably for some models (6.2%-12.1%). Certainty rate disparities were also pronounced, with larger models showing substantial degradations (22.0%-28.4%) on Global South claims.
  • Cost: OpenAI's GPT-40 offered a good balance between cost and performance ($2.22 per 1,000 claims), while smaller models were significantly cheaper but less accurate.

Limitations and Future Work

The authors acknowledge several limitations: the study relies on claims previously vetted by fact-checkers, potentially missing subtle misinformation. The temporal analysis, while valuable, may not capture rapidly evolving misinformation dynamics. The research focused on isolated LLM capabilities, not external augmentation (e.g., web search). Future work could explore how cultural epistemologies influence fact-checking, investigate multimodal misinformation, and develop methods for adversarial conditions. The paper also suggests the need for model optimization, subsidized access to advanced systems, or collaborative resource-sharing frameworks to address the "truth gap" and ensure equitable access to reliable AI fact-checking tools.

Summary updated

BibTeX
@misc{qazi2025scaling,
  author = {Ihsan Ayyub Qazi and Zohaib Khan and Abdullah Ghani and Agha Ali Raza and Zafar Ayyub Qazi and Wassay Sajjad and Ayesha Ali and Asher Javaid and Muhammad Abdullah Sohail and Abdul Hameed Azeemi},
  title = {{Scaling Truth: The Confidence Paradox in AI Fact-Checking}},
  year = {2025},
  eprint = {2509.08803},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.08803},
}
Figure