TabDSR: Decompose, Sanitize, and Reason for Complex Numerical Reasoning in Tabular Data

Changjiang Jiang, Fengchang Yu, Haihua Chen, Wei Lu, Jin ZengEMNLP 2025

TL;DR

This paper introduces TABDSR, a novel framework designed to improve the ability of Large Language Models (LLMs) to perform complex numerical reasoning on tabular data. By decomposing queries, sanitizing noisy tables, and employing a program-of-thoughts approach for reasoning, TABDSR significantly outperforms existing methods and introduces a new, challenging dataset (CalTab151) to mitigate data leakage issues in evaluations.

What Problem Does This Paper Address?

Complex numerical reasoning over tabular data is a critical task in many real-world applications, but current LLMs often struggle with it. This difficulty stems from several factors: (1) Multi-hop Complexity: Questions requiring multiple steps of reasoning are hard for LLMs to handle accurately. (2) Data Quality and Structure: Real-world tables can be noisy, with inconsistent data types, multi-level headers, and extraneous characters, leading to errors in automated processing. (3) Limited Numerical Computation: LLMs tend to mimic numerical procedures rather than truly compute, leading to inaccuracies. Existing datasets also suffer from data leakage, making it difficult to reliably assess model performance.

Methods

TABDSR is a three-agent framework inspired by human table reasoning:

  1. Query Decomposer Agent: This agent takes the user's question and breaks it down into smaller, tractable sub-questions. It focuses solely on the question text, using conjunctions and punctuation as delimiters, to avoid being influenced by table content and to ensure a more robust decomposition.
  2. Table Sanitizer Agent: This agent cleans and preprocesses the tabular data. It addresses structural issues like multi-level headers and segmented tables, and optimizes cell content by removing extraneous symbols, converting data to appropriate numerical types, and standardizing missing values. A reflection mechanism with Python parsing is used to ensure accuracy.
  3. PoT-based Reasoner Agent: This agent uses Program-of-Thoughts (PoT) to generate executable Python code. This code leverages the Pandas library to perform calculations on the sanitized table data, addressing the LLMs' inherent limitations in precise numerical computation. The pipeline ensures that the decomposed sub-questions and the clean data are fed into the reasoner for accurate final answer generation.

Datasets

The paper introduces CalTab151, a new dataset specifically designed for complex numerical reasoning over tables. It was constructed by:

  • Numerical Perturbation: Randomly perturbing cell values by ±3%-5%.
  • Cell Noise Addition: Injecting symbols like '$', '€', or '%' into numeric columns.
  • Structural Randomization: Shuffling rows/columns and randomly deleting some.
  • Random Null Value Filling: Replacing cells with labels like "None", "Null", "N/A", "???", or "-".
  • Multi-hop Question Generation: Creating complex questions requiring multiple steps.
  • Answer Annotation: Manual verification of all generated answers.

CalTab151 is composed of 151 table samples drawn from existing datasets like TableBench, FinQA, TAT-QA, and AitQa. The authors also used TAT-QA, TableBench, and CALTAB151 for experimental evaluation.

Key Insights and Ideas

  • Decomposition is Key: Breaking down complex questions into simpler sub-questions is crucial for LLMs to handle multi-hop reasoning accurately.
  • Data Quality Matters Immensely: Noisy and inconsistently formatted tables are a major bottleneck for numerical reasoning, and robust sanitization is essential.
  • PoT for True Computation: LLMs' inherent limitations in numerical computation can be overcome by generating and executing code, rather than relying on their learned patterns.
  • New Dataset for Rigorous Evaluation: CalTab151's construction, with its focus on numerical perturbation and noise, provides a more challenging and reliable benchmark, mitigating the issue of data leakage found in existing datasets.
  • Transferability Across Models: The prompt-based nature of TABDSR allows it to effectively enhance the performance of various LLMs, regardless of their size or architecture, demonstrating its general applicability.

Key Results

TABDSR consistently outperforms existing methods across TAT-QA, TableBench, and CALTab151. Notably, it achieved state-of-the-art (SOTA) performance with significant accuracy improvements:

  • TAT-QA: +8.79% improvement.
  • TableBench: +6.08% improvement.
  • CALTAB151: +19.87% improvement.

Even a 7B-parameter model using TABDSR surpassed larger models in some cases, highlighting the framework's efficiency. The ablation study showed that each agent (Decomposer, Sanitizer, Reasoner) contributes significantly to the overall performance, with the combined approach yielding the best results.

Limitations and Future Work

  • Human-Level Performance: TABDSR, while strong, still falls short of human performance in complex tabular reasoning, and its capabilities are limited by the underlying LLM's reasoning ability.
  • Dataset Annotation Cost: Manually verifying CalTab151 was costly, and the dataset size is relatively small with limited question types. Future work aims to expand CalTab151.
  • Restricted Decomposer: The current Decomposer agent operates solely on the question. Future work may explore hybrid approaches that incorporate table signals while maintaining decomposition reliability.
  • Sanitizer Reflection: The Sanitizer's reflection mechanism can lead to redundant calls if the base model struggles. Future work will implement call monitoring and fallback mechanisms.
  • Executor Failure Rate: The Executor agent has the highest failure rate due to the difficulty of generating robust code for diverse and noisy table inputs. This remains an open challenge.

Summary updated

BibTeX
@inproceedings{jiang2025tabdsr,
  author = {Changjiang Jiang and Fengchang Yu and Haihua Chen and Wei Lu and Jin Zeng},
  title = {{TabDSR: Decompose, Sanitize, and Reason for Complex Numerical Reasoning in Tabular Data}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure