TL;DR
This paper reframes Text-to-SQL training not merely as a query-generation task, but as a powerful scaffold for teaching large language models (LLMs) generalized tabular reasoning. By combining synthetic clause-level Chain-of-Thought (CoT) supervision with Group Relative Policy Optimization (GRPO), the authors demonstrate that models trained exclusively on Text-to-SQL data achieve massive performance boosts on completely separate, reasoning-intensive tabular QA datasets where SQL isn't even required.
What Problem Does This Paper Address?
While LLMs have become quite proficient at translating natural language into executable SQL queries, traditional training methods narrowly optimize for syntactic correctness or execution accuracy on specific databases. This often leads to overfitting, where models memorize schema-specific templates rather than learning how to truly understand and manipulate tabular data structures. Consequently, models fine-tuned purely for Text-to-SQL tend to perform poorly on broader, related tasks like table-based question answering (QA).
The authors address this gap by exploring whether SQL generation, when properly framed with reinforcement learning and intermediate reasoning steps, can serve as a foundational vehicle to induce emergent, transferable table-reasoning capabilities that generalize across entirely different datasets and tasks.
Methods
The authors propose a two-stage training framework designed to foster generalized tabular reasoning:
- Synthetic CoT Supervision (Supervised Fine-Tuning): The authors use OpenAI's
o3-minito synthesize detailed, clause-level natural language reasoning traces for Text-to-SQL questions. These traces explicitly teach the model how to traverse, filter, and aggregate table fields. A second LLM acts as a verifier to filter out incorrect or inconsistent traces, leaving a high-quality dataset of 3,500 verified triples (Question, CoT, SQL) used for initial supervised fine-tuning (SFT). - Reinforcement Learning via GRPO: To align execution accuracy with broader reasoning fidelity, they apply Group Relative Policy Optimization (GRPO), evaluating groups of candidate outputs to provide a denser learning signal. Instead of using a sparse binary reward (correct/incorrect execution), they design a multi-dimensional reward function combining:
- Execution-Based Reward: Uses an LLM to count orthographic changes between predicted and gold queries to provide a smoother gradient for near-correct predictions.
- String Matching Reward: Measures character-level overlap of the longest contiguous matching subsequence.
- Component-Level Matching Reward: Computes token-level F1 scores across specific SQL clauses (e.g.,
SELECT,WHERE,GROUP BY). - LLM Judge Reward: Categorizes outputs into ordinal quality classes (from Very Bad to Excellent).
Datasets
The framework is trained and evaluated using several key datasets:
- Clinton Dataset: A large-scale compilation of 26 individual datasets (including Spider, WikiSQL, ATIS, MIMICSQL, and eICU) spanning academic, medical, and government domains, used for the initial SFT-CoT phase.
- BIRD: A highly challenging, reasoning-intensive Text-to-SQL benchmark used during the GRPO reinforcement learning phase and for evaluation.
- CRT-QA: A complex tabular question-answering dataset used strictly for zero-shot evaluation to test cross-task generalization.
Key Insights and Ideas
- SQL as a Reasoning Scaffold: SQL's strict logical structure forces an LLM to systematically plan its data traversal. This structured constraint acts as an effective scaffold for learning robust data manipulation.
- Tabular "Aha-Moments": During GRPO training, the authors observed emergent, schema-aware reasoning. When evaluated on standard natural language tabular QA tasks, the model implicitly formulated internal, SQL-like logic to filter and compare rows to derive binary answers, even though no SQL output was requested.
- Asymmetric Gains for Compressed Models: Distilled and quantized models (like the 4-bit quantized distilled DeepSeek LLaMA 8B used in the paper) benefit significantly more from explicit CoT supervision (+33.9% relative gain on CRT-QA) than uncompressed models (+14.5% for Qwen). This suggests that explicit step-by-step guidance compensates for lost representational capacity in compressed architectures.
Key Results
- Cross-Task Generalization: On the zero-shot tabular QA benchmark (CRT-QA), the combined SFT-CoT-GRPO methodology yielded dramatic improvements. The LLaMA model jumped from a base accuracy of 41.3% to 55.3% (a 33.9% relative increase), while the Qwen model improved from 49.0% to 56.1% (a 14.5% relative increase).
- Text-to-SQL Performance: On the highly competitive BIRD benchmark, the GRPO-trained models showed substantial gains over their base versions, with Qwen reaching 15.5% execution accuracy under the best four-rewards configuration.
- Reward Synergy: Ablation studies revealed that relying exclusively on string matching leads to diminished performance in later training stages. Combining continuous string matching with discrete semantic evaluations (the LLM Judge Reward) provided the most robust optimization path.
Limitations and Future Work
- Model Scale: The study focused on medium-scale foundation models (7B/8B parameters). The authors note that these modest parameter counts may limit performance on tasks requiring deep domain expertise, such as biomedical or legal reasoning.
- Execution Proxy: Due to partial database constraints, the authors relied on
o3-minias an automated judge and execution proxy rather than evaluating against a live database management system (DBMS). They plan to incorporate actual DBMS-based evaluation in future iterations. - Pipeline Simplicity: The current framework employs only two training stages, whereas state-of-the-art reasoning pipelines (like DeepSeek-R1) leverage up to four stages, including pre-CoT bootstrapping and iterative refinement. The authors suggest that adding these stages could further enhance robustness.
Summary updated