Abstract
Structured tabular data is a fundamental data type in numerous fields, and the capacity to reason over tables is crucial for answering questions and validating hypotheses. However, constructing labeled data for complex reasoning tasks is labor intensive, and the quantity of annotated data remains insufficient to support the intricate demands of real-world applications. To address the insufficient annotation challenge, we present a self-training framework for unsupervised complex tabular reasoning (UCTR-ST) by generating diverse synthetic data with complex logic. Specifically, UCTR-ST incorporates several essential techniques: we aggregate diverse programs and execute them on tables based on a "Program-Management" component, and we bridge the gap between programs and text with a powerful "Program-Transformation" module that generates natural language sentences with complex logic. Furthermore, we optimize the procedure using a "Table-Text Manipulator" to handle joint table-text reasoning scenarios. The entire framework utilizes self-training techniques to leverage the unlabeled training data, which results in significant performance improvements when tested on real-world data. Experimental results demonstrate that UCTRST achieves above 90% of the supervised model performance on different tasks and domains, reducing the dependence on manual annotation. Additionally, our approach can serve as a data augmentation technique, significantly boosting the performance of supervised models in low-resourced domains.
1. Introduction
Structured tabular data is ubiquitous across domains such as finance, science, and public administration. Reasoning over these tables is essential for tasks like question answering and fact verification. However, complex reasoning—involving multi-step logic, arithmetic, and cross-referencing—requires high-quality labeled data, which is notoriously difficult and expensive to produce manually. This "data drought" limits the scalability of Large Language Models (LLMs) in specialized tabular domains. This paper introduces UCTR-ST, a framework designed to eliminate the need for manual labels by creating a pipeline that can teach itself how to reason over any table it encounters.
2. The UCTR-ST Framework
To mitigate the reliance on manual labels, we propose the Unsupervised Complex Tabular Reasoning via Self-Training (UCTR-ST) framework. This approach leverages logical programs to generate high-fidelity synthetic training data and employs a self-training loop to adapt to real-world data distributions.
2.1 Synthetic Data Generation
The core of UCTR-ST lies in its ability to create diverse and logically rigorous training samples. This is achieved through three primary modules:
- Program-Management: This module samples formal programs (e.g., SQL, logical forms, or arithmetic expressions) and executes them against raw tables to derive ground-truth answers.
- Program-Transformation: Generative models (such as GPT-2 or BART) are utilized to "verbalize" these programs into natural language questions or claims. For example, a SQL
SUMoperation is transformed into a question like "What is the total cost for 2023?" - Table-Text Manipulator: To simulate real-world contexts where tables are embedded in text, this module uses Table-to-Text and Text-to-Table operators to create hybrid reasoning samples.
2.2 Iterative Self-Training
The framework employs a Teacher-Student architecture. Initially, a model is trained on synthetic data. This model (the Teacher) then generates pseudo-labels for unlabeled real-world data. The most confident predictions are used to augment the training set for the Student model. This iterative process allows the system to bridge the gap between the "clean" synthetic distribution and the "messy" reality of human-authored queries.
3. Experimental Evaluation
UCTR-ST was evaluated on four major benchmarks covering different domains: FEVEROUS (fact verification), TAT-QA (finance QA), WiKiSQL (text-to-SQL), and SEM-TAB-FACTS (scientific fact verification).
- Closing the Gap: The framework consistently achieved over 90% of the performance of fully supervised models. For instance, on WiKiSQL, it reached 62.7% denotation accuracy without any manual labels.
- Superior to Baselines: UCTR-ST significantly outperformed previous unsupervised state-of-the-art methods. On the finance-heavy TAT-QA dataset, it achieved a 47.6 F1 score, compared to 27.7 for the MQA-QG baseline.
- Low-Resource Powerhouse: When human labels are scarce (e.g., only 50 samples), UCTR-ST serves as a powerful data augmentation tool, often doubling the performance of baseline models.
4. Conclusion and Future Work
UCTR-ST demonstrates that formal logical programs can serve as a robust backbone for unsupervised learning in complex reasoning tasks. By bridging the gap between structured data and natural language, the framework reduces the need for labor-intensive annotation. Future research will focus on expanding the diversity of supported reasoning types and developing automated methods for table-specific program generation that can adapt to a table's specific data distribution.
Summary updated