When a user asks a database a question like "Which departments have a budget higher than the average salary of their employees?", a Text-to-SQL system must perform a complex cognitive leap. It cannot simply translate words into syntax; it must map ambiguous natural language tokens ("departments", "budget", "average salary") to precise structural elements (tables, columns, primary keys, and foreign keys) and then synthesize them into a syntactically valid relational query.
Historically, this mapping was handled by modular architectures or generative parsers paired with discriminative rerankers, as explored by Alessandra Giordani et al. in Translating Questions to SQL Queries with Generative Parsers Discriminatively Reranked. Later systems simplified these architectures by framing query generation as machine reading comprehension (Zeyu Yan et al. in SQL Generation via Machine Reading Comprehension) or utilized pointer-generator networks to track context across multi-turn sessions (Huajie Wang et al. in PG-GSQL: Pointer-Generator Network with Guide Decoding for Cross-Domain Context-Dependent Text-to-SQL Generation).
However, modern large-scale databases present a massive bottleneck: schema complexity. When a database contains hundreds of tables and thousands of columns, feeding the entire schema into a Large Language Model (LLM) prompt introduces severe noise, token overhead, and structural blindness. The modern research direction focuses on solving this through two main pillars: scalable schema linking and structured tabular reasoning.
Pillar 1: Scalable and Graph-Aware Schema Linking
Schema linking is the process of identifying exactly which tables and columns are required to answer a specific query. Recent breakthroughs treat schema linking not as a simple text-matching problem, but as an optimization and graph-induction challenge.
Optimizing Relevance vs. Redundancy
If a prompt includes too few columns, the model cannot write the query; if it includes too many, the model gets confused by irrelevant distractors. To solve this trade-off, Zheng Yuan et al. introduced KaSLA in Knapsack Schema Linking Agent for LLM-Based Text-to-SQL Generation. They frame schema linking as a classic knapsack problem, explicitly optimizing for maximum semantic relevance while strictly minimizing redundancy to keep prompts clean and effective.
Ensuring Topological Connectivity
A common failure mode in multi-table joins is "structural blindness"—the model selects the correct endpoints (e.g., Employee.name and Department.budget) but fails to find the intermediate junction tables needed to connect them. Because relational databases are fundamentally graphs where tables are nodes and foreign keys are edges, schema retrieval must respect this topology.
Jianwei Zhong et al. addressed this in Graph-Link: Bridging the Semantic-Structural Gap in Text-to-SQL via Constrained Subgraph Induction by reformulating schema linking as a constrained subgraph induction problem. By using a Steiner-tree-based optimization, they guarantee that the retrieved schema elements maintain topological connectivity, boosting accuracy on complex multi-hop queries. Similarly, Changzhe Ma et al. in Integrating Relational Structure to Heterogeneous Graph for Chinese NL2SQL Parsers utilize heterogeneous graphs and relative position attention to resolve language ambiguity and schema complexity simultaneously.
Scaling to Industrial Databases
When databases scale to thousands of columns, even reading the full schema once exceeds standard context windows. To handle this, Zhongyu Wang et al. proposed AutoLink in AutoLink: Autonomous Schema Exploration and Expansion for Scalable Schema Linking in Text-to-SQL at Scale. Instead of analyzing the full schema statically, an autonomous agent iteratively explores and expands a relevant schema subset. This allows systems to maintain high recall on massive industrial databases while consuming minimal tokens.
This is complemented by methods like GRIT by Yujin Kang et al. in GRIT: Guided Relational Integration for Efficient Multi-Table Understanding, which efficiently encodes relational structures into LLM-friendly prompts, and syntax-aware dense retrieval mechanisms designed by Nengzheng Jin et al. in Enhancing Open-Domain Table Question Answering via Syntax- and Structure-aware Dense Retrieval to align human phrasing with database elements.
Pillar 2: Deep Tabular Reasoning and Execution Reliability
Once the relevant schema is isolated, the next challenge is execution. Generating a valid SQL query requires a deep understanding of tabular logic, operations (like aggregations, groupings, and nested subqueries), and execution safety.
Reinforcement Learning and Chain-of-Thought
Rather than relying on raw text generation, recent frameworks train models to "think" about tables. Josefa Lia Stoisser et al. developed a framework in Sparks of Tabular Reasoning via Text2SQL Reinforcement Learning that pairs synthetic Chain-of-Thought (CoT) traces with Group Relative Policy Optimization (GRPO). By reinforcing the model's underlying reasoning steps rather than just the final string output, they train the model to understand why a specific clause is needed, unlocking a deeper generalization capability over structured data.
Multi-Turn Interactions and Tool Use
Real-world database interaction is rarely a single-shot success. G.J. Xiong et al. introduced Interactive-T2S in Multi-Turn Interactions for Text-to-SQL with Large Language Models, which allows LLMs to interact dynamically with the database using a set of general tools. By executing intermediate steps and reasoning through multi-turn feedback loops, the system can self-correct errors and handle wide tables efficiently without relying on external knowledge.
Verification, Abstention, and Templates
In high-stakes domains like healthcare, finance, or meteorology (e.g., Jinkyung Jo et al. in An Integrated Search System for Korea Weather Data), an incorrect SQL query can be catastrophic. Systems must know when they do not know the answer.
Jerrin Thomas et al. tackled this in LTRC-IIITH at EHRSQL 2024: Enhancing Reliability of Text-to-SQL Systems through Abstention and Confidence Thresholding by integrating an explicit abstention module and confidence thresholding to filter out unanswerable questions or high-risk errors. For interpretability, systems like TURING (Peng Xu et al. in TURING: an Accurate and Interpretable Multi-Hypothesis Cross-Domain Natural Language Database Interface) generate multiple hypotheses alongside step-by-step natural language explanations, allowing users to verify the query logic before execution.
Alternatively, for cost-effective and privacy-compliant workflows, Ye Yuan et al. developed FACTS in FACTS: Fast, Accurate, and Privacy-Compliant Table Summarization via Offline Template Generation, which automates the creation of reusable offline templates consisting of SQL queries and formatting, avoiding the need to expose sensitive data directly to an LLM at inference time.
Expanding Beyond SQL: Structured RAG and Code Generation
The principles of Text-to-SQL are now expanding into broader tabular question-answering paradigms. For instance, Anshuman Aryan et al. in AlphaPro at SemEval-2025 Task 8: A Code Generation Approach for Question-Answering over Tabular Data demonstrate that transforming questions into schema-aware paraphrases and generating executable Python code can be highly effective for tabular reasoning, bypassing SQL entirely when programmatic data manipulation is preferred.
Furthermore, the boundary between unstructured text and structured databases is blurring. In Structured Retrieval-Augmented Generation for Multi-Doc Multi-Entity Question Answering, Teng Lin et al. introduce SRAG, a framework that converts unstructured multi-document text into structured relational tables on the fly, and then guides an LLM to reason over those tables using SQL.
By combining precise graph-based schema linking, interactive execution tools, and robust reinforcement learning, modern Text-to-SQL systems have evolved from fragile syntax translators into resilient, scalable interfaces capable of deep reasoning over structured knowledge.
Key Foundational Work
Clause-Wise and Recursive Decoding for Complex and Cross-Domain Text-to-SQL Generation (2019) — contributed 1 concept elements to this direction.
Translating Questions to SQL Queries with Generative Parsers Discriminatively Reranked (2012) — contributed 1 concept elements to this direction.