Weaver: Interweaving SQL and LLM for Table Reasoning

Rohit Khoja, Devanshu Gupta, Yanjie Fu, Dan Roth, Vivek Gupta

Abstract

Querying tables with unstructured data is challenging due to the presence of text (or image), either embedded in the table or in external paragraphs, which traditional SQL struggles to process, especially for tasks requiring semantic reasoning. While Large Language Models (LLMs) excel at understanding context, they face limitations with long input sequences. Existing approaches that combine SQL and LLM typically rely on rigid, predefined workflows, limiting their adaptability to complex queries. To address these issues, we introduce Weaver, a modular pipeline that dynamically integrates SQL and LLM for table-based question answering (Table QA). Weaver generates a flexible, step-by-step plan that combines SQL for structured data retrieval with LLMs for semantic processing. By decomposing complex queries into manageable subtasks, Weaver improves accuracy and generalization. Our experiments show that consistently outperforms state-of-the-art methods across four Table QA datasets, reducing both API calls and error rates.

Introduction

Real-world tables are rarely composed of clean, structured data alone. They frequently contain "messy" elements such as free-form text, images, or implicit relationships that require external knowledge to interpret. Traditional tools for table reasoning face a "Goldilocks" problem: SQL and Python are excellent for logical grouping and numerical calculations but are "blind" to semantic context. Conversely, Large Language Models (LLMs) possess vast world knowledge and can interpret text but struggle with long contexts, complex multi-step logic, and precise numerical reasoning over large datasets.

Existing hybrid approaches often employ rigid, predefined workflows or treat LLMs as simple function calls within SQL. These methods are limited when handling complex, multi-hop queries that require alternating between logical and semantic steps. Weaver addresses these challenges by introducing a modular framework that dynamically interweaves SQL for structured operations and LLMs for semantic reasoning.

Methodology: The Weaver Pipeline

Weaver treats table reasoning as a structured planning and execution problem. The pipeline consists of five distinct phases designed to maximize both logical precision and semantic depth:

  1. Pre-processing: This phase cleans table headers to avoid SQL reserved-word conflicts and utilizes an LLM to generate descriptive metadata for columns, providing context for subsequent steps.
  2. Planning: An LLM generates a flexible, multi-step reasoning plan. Each subtask is explicitly categorized as either a SQL Step (for filtering, sorting, or aggregating) or an LLM Step (for semantic inference or data augmentation, such as identifying a driver's nationality from their name).
  3. Plan Verification: A secondary LLM check ensures the plan is logically sound and can handle potential data inconsistencies before execution begins.
  4. Code Execution: The pipeline executes the plan sequentially, creating intermediate tables at each step. For LLM-based steps, Weaver uses "Column-Wise Batching" to process data in manageable chunks, preventing context window overflows.
  5. Answer Extraction: A final LLM step retrieves the specific answer from the fully processed table.
The Weaver pipeline overview showing the interweaving of SQL and LLM steps.

Optimization and Efficiency

To maintain efficiency and reduce costs, Weaver employs sophisticated Query Plan Optimization. The system reorders steps to "push down" SQL filters as early as possible. By pruning the table with SQL before sending data to the LLM, Weaver minimizes expensive API calls and reduces latency. Additionally, Weaver can merge sequential SQL operations into single, complex queries and process LLM steps in parallel via data chunking.

Optimization strategies including SQL step push-down and parallel LLM processing.

Experimental Results

Weaver was evaluated on hybrid versions of established benchmarks, including WikiTQ (short-form QA), TabFact (fact-checking), and FinQA (numerical reasoning), as well as multimodal datasets (FinQA-MM, OTT-QA-MM, and MMTabQA-MM).

  • Accuracy: Weaver consistently outperformed strong baselines like Binder and ProTrix. On the WikiTQ-hybrid set, Weaver surpassed ProTrix by 5.5% across various models. On TabFact, it achieved a breakthrough 91.2% accuracy using DeepSeek-R1.
  • Efficiency: Weaver is remarkably lean, averaging only 5-6 API calls per query. In contrast, self-consistency methods like Binder can require 50-60 calls to reach a stable answer.
  • Robustness: In tests on large tables (30+ rows), Weaver maintained an accuracy of 65.6%, while competing methods saw performance drop significantly (e.g., ProTrix falling to 37.5%).

Conclusion and Future Work

Weaver demonstrates that decoupling logical planning from execution and providing intermediate traceability significantly improves table reasoning. While currently optimized for English-language single tables, future work will focus on extending the framework to multi-table relational databases and native support for hierarchical table structures. The integration of Vision-Language Models (VLMs) is also planned to enhance performance on charts and embedded images.

Summary updated

Abstract

Querying tables with unstructured data is challenging due to the presence of text (or image), either embedded in the table or in external paragraphs, which traditional SQL struggles to process, especially for tasks requiring semantic reasoning. While Large Language Models (LLMs) excel at understanding context, they face limitations with long input sequences. Existing approaches that combine SQL and LLM typically rely on rigid, predefined workflows, limiting their adaptability to complex queries. To address these issues, we introduce Weaver, a modular pipeline that dynamically integrates SQL and LLM for table-based question answering (Table QA). Weaver generates a flexible, step-by-step plan that combines SQL for structured data retrieval with LLMs for semantic processing. By decomposing complex queries into manageable subtasks, Weaver improves accuracy and generalization. Our experiments show that consistently outperforms state-of-the-art methods across four Table QA datasets, reducing both API calls and error rates.
BibTeX
@inproceedings{khoja2025weaver,
  author = {Rohit Khoja and Devanshu Gupta and Yanjie Fu and Dan Roth and Vivek Gupta},
  title = {{Weaver: Interweaving SQL and LLM for Table Reasoning}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=VuOmPU4nLW},
}
Figure