Introduction
Traditional Natural Language to SQL (NL2SQL) research typically operates under the assumption of a "perfect world": users provide clear, specific questions, and a complete database schema is available for execution. However, real-world developer interactions—such as those found on Stack Overflow—are often messy and "under-specified." Developers frequently seek help with SQL logic or syntax templates without providing specific table names or schemas. In these scenarios, generating a 100% executable query is impossible and often unnecessary.
This paper introduces the task of NL2pSQL, which focuses on generating human-readable Pseudo-SQL (pSQL). These queries serve as templates to guide developers, prioritizing structural correctness and logic over immediate executability.
The Challenge of Under-Specified Queries
On platforms like Stack Overflow, a question might be as vague as "How do I copy from one table to another?" A traditional NL2SQL model would struggle without a schema. NL2pSQL addresses this by producing a well-formed template like INSERT INTO table1 SELECT * FROM table2.
To support this research, the authors curated the NL2pSQL Dataset, consisting of 1,244 pairs of natural language questions and SQL answers crawled from Stack Overflow. Unlike standard datasets like WikiSQL, which focus on simple SELECT statements, this dataset covers 362 distinct SQL keywords, reflecting the diverse needs of professional developers.
Proposed Architecture: The Denoising Pipeline
The core technical contribution is a two-stage generative architecture designed to ensure that the output is not only relevant but also syntactically valid (well-formed).
- Initial Generation: A base sequence-to-sequence (Seq2Seq) or CopyNet model translates the natural language input into an initial SQL statement.
- Denoising Autoencoder (DAE): Generative models often produce "noisy" SQL characterized by repetitive tokens or syntax errors. The authors introduced a DAE trained on a massive synthetic dataset of 406,384 perturbed SQL pairs. This layer acts as a "syntax fixer," taking the potentially ill-formed output from the first stage and refining it into a well-formed statement that passes a SQL parser.
Evaluation Metrics
Because standard BLEU scores can be misleading for code—where a logically perfect query might use different variable names than the ground truth—the authors proposed two new metrics:
- Canonical-BLEU: Replaces identifiers with indexed placeholders to focus on logic.
- SQL-BLEU: Measures structural accuracy by calculating scores based solely on SQL keywords.
Key Results
The addition of the Denoising Autoencoder yielded significant improvements in the "well-formedness" of the generated queries. As shown in the experimental results, the percentage of parsable queries increased by as much as 43% in certain configurations (e.g., Seq2Seq Char-based models).
Crucially, while the DAE drastically improved the syntax and readability of the SQL, the underlying BLEU scores remained stable. This indicates that the denoising process successfully repairs the "form" of the query without distorting the "meaning" captured during the initial translation phase.
Conclusion
NL2pSQL shifts the focus of natural language interfaces for databases from strict execution to developer support. By cascading a standard generator with a denoising autoencoder, the system provides a robust tool for generating high-quality SQL templates from the kind of under-specified questions common in real-world programming environments. Future work aims to expand this approach to multiple SQL dialects and incorporate broader conversational context from developer forums.
Summary updated