Sparks of Tabular Reasoning via Text2SQL Reinforcement Learning

Josefa Lia Stoisser, Marc Boubnovski Martell, Julien Fauqueur

TL;DR

This paper reframes Text-to-SQL training not merely as a query-generation task, but as a powerful scaffold for teaching large language models (LLMs) generalized tabular reasoning. By combining synthetic clause-level Chain-of-Thought (CoT) supervision with Group Relative Policy Optimization (GRPO), the authors demonstrate that models trained exclusively on Text-to-SQL data achieve massive performance boosts on completely separate, reasoning-intensive tabular QA datasets where SQL isn't even required.

What Problem Does This Paper Address?

While LLMs have become quite proficient at translating natural language into executable SQL queries, traditional training methods narrowly optimize for syntactic correctness or execution accuracy on specific databases. This often leads to overfitting, where models memorize schema-specific templates rather than learning how to truly understand and manipulate tabular data structures. Consequently, models fine-tuned purely for Text-to-SQL tend to perform poorly on broader, related tasks like table-based question answering (QA).

The authors address this gap by exploring whether SQL generation, when properly framed with reinforcement learning and intermediate reasoning steps, can serve as a foundational vehicle to induce emergent, transferable table-reasoning capabilities that generalize across entirely different datasets and tasks.

Methods

The authors propose a two-stage training framework designed to foster generalized tabular reasoning:

  1. Synthetic CoT Supervision (Supervised Fine-Tuning): The authors use OpenAI's o3-mini to synthesize detailed, clause-level natural language reasoning traces for Text-to-SQL questions. These traces explicitly teach the model how to traverse, filter, and aggregate table fields. A second LLM acts as a verifier to filter out incorrect or inconsistent traces, leaving a high-quality dataset of 3,500 verified triples (Question, CoT, SQL) used for initial supervised fine-tuning (SFT).
  2. Reinforcement Learning via GRPO: To align execution accuracy with broader reasoning fidelity, they apply Group Relative Policy Optimization (GRPO), evaluating groups of candidate outputs to provide a denser learning signal. Instead of using a sparse binary reward (correct/incorrect execution), they design a multi-dimensional reward function combining:
  3. Execution-Based Reward: Uses an LLM to count orthographic changes between predicted and gold queries to provide a smoother gradient for near-correct predictions.
  4. String Matching Reward: Measures character-level overlap of the longest contiguous matching subsequence.
  5. Component-Level Matching Reward: Computes token-level F1 scores across specific SQL clauses (e.g., SELECT, WHERE, GROUP BY).
  6. LLM Judge Reward: Categorizes outputs into ordinal quality classes (from Very Bad to Excellent).
Overview of the training pipeline

Datasets

The framework is trained and evaluated using several key datasets:

  • Clinton Dataset: A large-scale compilation of 26 individual datasets (including Spider, WikiSQL, ATIS, MIMICSQL, and eICU) spanning academic, medical, and government domains, used for the initial SFT-CoT phase.
  • BIRD: A highly challenging, reasoning-intensive Text-to-SQL benchmark used during the GRPO reinforcement learning phase and for evaluation.
  • CRT-QA: A complex tabular question-answering dataset used strictly for zero-shot evaluation to test cross-task generalization.

Key Insights and Ideas

  • SQL as a Reasoning Scaffold: SQL's strict logical structure forces an LLM to systematically plan its data traversal. This structured constraint acts as an effective scaffold for learning robust data manipulation.
  • Tabular "Aha-Moments": During GRPO training, the authors observed emergent, schema-aware reasoning. When evaluated on standard natural language tabular QA tasks, the model implicitly formulated internal, SQL-like logic to filter and compare rows to derive binary answers, even though no SQL output was requested.
  • Asymmetric Gains for Compressed Models: Distilled and quantized models (like the 4-bit quantized distilled DeepSeek LLaMA 8B used in the paper) benefit significantly more from explicit CoT supervision (+33.9% relative gain on CRT-QA) than uncompressed models (+14.5% for Qwen). This suggests that explicit step-by-step guidance compensates for lost representational capacity in compressed architectures.
Training on Text-to-SQL, Evaluating on Dual Tasks

Key Results

  • Cross-Task Generalization: On the zero-shot tabular QA benchmark (CRT-QA), the combined SFT-CoT-GRPO methodology yielded dramatic improvements. The LLaMA model jumped from a base accuracy of 41.3% to 55.3% (a 33.9% relative increase), while the Qwen model improved from 49.0% to 56.1% (a 14.5% relative increase).
  • Text-to-SQL Performance: On the highly competitive BIRD benchmark, the GRPO-trained models showed substantial gains over their base versions, with Qwen reaching 15.5% execution accuracy under the best four-rewards configuration.
  • Reward Synergy: Ablation studies revealed that relying exclusively on string matching leads to diminished performance in later training stages. Combining continuous string matching with discrete semantic evaluations (the LLM Judge Reward) provided the most robust optimization path.

Limitations and Future Work

  • Model Scale: The study focused on medium-scale foundation models (7B/8B parameters). The authors note that these modest parameter counts may limit performance on tasks requiring deep domain expertise, such as biomedical or legal reasoning.
  • Execution Proxy: Due to partial database constraints, the authors relied on o3-mini as an automated judge and execution proxy rather than evaluating against a live database management system (DBMS). They plan to incorporate actual DBMS-based evaluation in future iterations.
  • Pipeline Simplicity: The current framework employs only two training stages, whereas state-of-the-art reasoning pipelines (like DeepSeek-R1) leverage up to four stages, including pre-CoT bootstrapping and iterative refinement. The authors suggest that adding these stages could further enhance robustness.

Summary updated

Abstract

This work reframes the Text-to-SQL task as a pathway for teaching large language models (LLMs) to reason over and manipulate tabular data--moving beyond the traditional focus on query generation. We propose a two-stage framework that leverages SQL supervision to develop transferable table reasoning capabilities. First, we synthesize detailed chain-of-thought (CoT) traces from real-world SQL queries, providing step-by-step, clause-level supervision that teaches the model how to traverse, filter, and aggregate table fields. Second, we introduce a Group Relative Policy Optimization (GRPO) reinforcement learning objective that connects SQL execution accuracy to generalizable reasoning by encouraging steps that extend beyond task-specific syntax and transfer across datasets. Empirically, our approach improves performance on standard Text-to-SQL benchmarks and achieves substantial gains on reasoning-intensive datasets such as BIRD and CRT-QA, demonstrating enhanced generalization and interpretability. Specifically, the distilled-quantized LLaMA model achieved a relative 33.9\% increase in accuracy when trained on Text-to-SQL tasks, while Qwen achieved a relative 14.5\% increase. These results suggest that SQL can serve not only as a target formalism but also as an effective scaffold for learning robust, transferable reasoning over structured data.
BibTeX
@inproceedings{stoisser2025sparks,
  author = {Josefa Lia Stoisser and Marc Boubnovski Martell and Julien Fauqueur},
  title = {{Sparks of Tabular Reasoning via Text2SQL Reinforcement Learning}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}
Figure