A Multi-Agent Approach for Claim Verification from Tabular Data Documents

Rudra Ranajee Saha, Laks V. S. Lakshmanan, Raymond T. NgarXiv 2026 · arXiv

TL;DR

The paper introduces MACE (Multi-Agentic framework for Claim vErification), a novel prompt-based system that uses three specialized LLM agents—a Planner, an Executor, and a Verifier—to verify claims against tabular documents without any fine-tuning. By introducing a bidirectional feedback loop between agents, MACE allows significantly smaller open-source models (27B–92B parameters) to match or exceed the performance of massive 235B+ parameter models, drastically reducing memory requirements while providing fully transparent, step-by-step reasoning.

What Problem Does This Paper Address?

Verifying factual claims against structured data like tables embedded in financial reports, scientific papers, or corporate disclosures is a massive challenge for natural language processing. Unlike plain text, tables contain hierarchical headers, nested structures, and complex numerical relationships that require multi-step mathematical reasoning.

Current state-of-the-art approaches typically rely on massive, expensive proprietary models (like GPT-4) or require intensive task-specific pre-training and fine-tuning. Furthermore, existing systems that break the problem down into subtasks usually do so in a unidirectional pipeline. If the planning stage makes a minor error or misinterprets a table header, that error propagates unchecked through the extraction and computation stages, leading to an incorrect final verdict.

Methods

The authors propose MACE, a structured, zero-shot Chain-of-Thought (CoT) multi-agent framework built using the AutoGen library. Instead of relying on a single model to do everything, MACE splits the cognitive load across three specialized roles:

  1. Planner Agent: Analyzes the claim and evidence to devise a step-by-step verification strategy. It is specifically prompted to handle negations, resolve ambiguous qualifiers (e.g., "significant increase") by establishing contextual thresholds, and break down compound claims.
  2. Executor Agent: Implements the Planner's strategy. It extracts the exact cells from the table, performs the necessary mathematical calculations, and generates a provisional verdict (support, refute, or not enough info).
  3. Verifier Agent: Acts as quality control. It audits the Executor’s math and logic to ensure the explanation genuinely supports the verdict.

The Key Innovation: Bidirectional Feedback Loops Unlike rigid pipelines, MACE allows agents to talk back. If the Executor encounters an ambiguous instruction, it can output a "revise" request to the Planner. Similarly, if the Verifier catches a logical inconsistency or a mathematical error, it sends the task back to the Executor with specific correction requests.

Datasets

The framework was evaluated across both closed-domain (claims paired with a specific table) and open-domain (claims paired with a corpus where the correct table must first be retrieved) settings:

  • SciTab & SEM-TAB-FACTS (SemTab): Standard closed-domain benchmarks featuring complex reasoning over scientific tables.
  • FinDVer: An open-domain dataset consisting of long, hybrid-content financial documents.
  • SciTab-OD: A new open-domain benchmark engineered by the authors. They adapted the original SciTab dataset into an open-domain setting by unpairing the claims and using OpenAI's text-embedding-3 to retrieve the top relevant tables from a corpus.

Key Insights and Ideas

  • Collaboration Compensates for Scale: The structured multi-agent setup acts as a force multiplier for smaller models. For instance, the 8-billion-parameter Llama-8B model saw its performance jump by up to 54.2% on SemTab when deployed within MACE compared to standard single-model inference.
  • The Independent Verifier Paradox: The authors experimented with using an independent model (GPT-20B) as the Verifier versus using the same base model family across all three agents. They discovered that while an independent verifier drastically helps smaller models (like Mistral-7B), it actually hurts larger models (like Qwen-72B/235B). For advanced models, maintaining reasoning consistency within the same model family outweighs the benefit of an outside auditor, which often lacks the domain knowledge of the larger base model.
  • The Accuracy vs. Latency Trade-off: High accuracy and explainability come at a computational cost. MACE takes significantly longer to run than standard single-model Chain-of-Thought prompting due to sequential agent discussions and revision cycles.

Key Results

  • State-of-the-Art Performance: MACE achieved SOTA results on SciTab (0.71 Macro F1) and SemTab (0.90 Micro F1) using Qwen-235B.
  • Outperforming GPT-4o: On the open-domain SciTab-OD dataset, MACE outperformed prominent models like GPT-4o and Claude-3.5-Sonnet by 7.04%, achieving a Macro F1 of 0.76.
  • Massive Memory Savings: Smaller configurations of MACE (27B–92B total parameters) achieved 79.5% to 100% of the performance of the massive Qwen-235B baseline while consuming only 11.5% to 39.1% of the memory footprint.

Limitations and Future Work

The primary limitation of MACE is its runtime overhead. Because agents must converse sequentially and occasionally loop back for revisions, execution times can be 5.3× to 27.5× longer than single-model inference, making it less suitable for real-time, latency-sensitive applications. Additionally, the framework does not yet include specialized preprocessing strategies for handling extremely large tables that exceed an LLM's context window.

For future work, the authors plan to introduce specialized agents dedicated solely to advanced mathematical reasoning and to implement feedback loops that connect the "not enough info" (NEI) predictions directly back to the retrieval module to fetch better evidence.

Summary updated

Abstract

We present a novel approach for claim verification from tabular data documents. Recent LLM-based approaches either employ complex pretraining/fine-tuning or decompose verification into subtasks, often lacking comprehensive explanations and generalizability. To address these limitations, we propose a Multi-Agentic framework for Claim verification (MACE) consisting of three specialized agents: Planner, Executor, and Verifier. Instead of elaborate finetuning, each agent employs a zero-shot Chain-of-Thought setup to perform its tasks. MACE produces interpretable verification traces, with the Planner generating explicit reasoning strategies, the Executor providing detailed computation steps, and the Verifier validating the logic. Experiments demonstrate that MACE achieves state-of-the-art (SOTA) performance on two datasets and performs on par with the best models on two others, while achieving 80--100\% of best performance with substantially smaller models: 27--92B parameters versus 235B. This combination of competitive performance, memory efficiency, and transparent reasoning highlights our framework's effectiveness.
BibTeX
@misc{saha2026multi,
  author = {Rudra Ranajee Saha and Laks V. S. Lakshmanan and Raymond T. Ng},
  title = {{A Multi-Agent Approach for Claim Verification from Tabular Data Documents}},
  year = {2026},
  eprint = {2604.17225},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.17225},
}

Loading related content...

Figure