Gate Each Piece Before Assembly: Self-Checking Sub-Query Generation in Small Text-to-SQL Models

When a language model writes a complicated database query, it often builds it from smaller pieces: a subquery here, a filter there, then a final statement that stitches them together. Recent work shows that writing the pieces first helps compact models reach the accuracy of much larger ones. It also shows a predictable failure. If one piece is wrong, say a filter on the wrong column, the model copies that error into the final query. Nothing downstream ever questions it. We propose to train a single 7-billion-parameter model to stop after each piece and output a short verdict: "this piece is fine" or "this piece is wrong, and here is the kind of error." Only a flagged piece is regenerated. The final query is assembled from pieces that passed. The check runs without querying the database at inference time. It costs one extra token per piece, rather than a second model or a full rewrite of the query.

The opportunity is that three findings in this literature have not been combined. Hierarchical decomposition fixes structural dead-ends but spreads hallucinated pieces into the final query. Post-hoc error detectors with explicit error vocabularies work, but only as multi-model pipelines around frontier models. Small specialized models can learn narrow skills very well. The open question is whether a small parser can also act as its own reliable checker, but only at the boundaries between pieces, where checking is easiest. A whole finished query is much harder to inspect.

Core Claim

Our central bet is about where checking is easy. Judging a short, self-contained sub-query against the question and schema is a much easier discrimination problem than judging a monolithic query. A small model can learn the easier problem well enough to break what Addressing Semantic Blind Spots in Text-to-SQL via Component Pre-generation and AST Matching Rewards calls the "hallucination chain."

That paper pre-generates modular "SQL Units" (isolated WHERE clauses, nested subqueries) and trains with a reward based on matching abstract syntax trees, the tree form of a program's structure. It reports 65.78% execution accuracy on BIRD-dev. Its authors also identify that an erroneous unit is faithfully absorbed into the final query. Our model differs in one ingredient: a learned accept/reject verdict at each unit boundary, with localized resampling. Our claim is falsifiable. If the gate is no better than resampling whole queries and majority-voting at the same token budget, the idea fails.

Research Program

The parser is built on Qwen2.5-Coder-7B, the base model used by Reward-SQL. Reward-SQL decomposes SQL into a chain of Common Table Expressions (named intermediate subqueries), scores each one with a separately trained process reward model, and uses that model to guide reinforcement learning and best-of-N reranking. That process reward model executes intermediate subqueries against the database.

Our model instead emits, after each unit, one of a small set of verdict tokens. One token means no error. The others name an error class, modeled on the vocabulary of ErrorLLM. ErrorLLM uses dedicated error tokens and separate localization and refinement models on top of GPT-4o, and shows that unstructured self-correction corrupts valid queries. We keep its vocabulary but fold detection into the generating model's own decoder. Localization is unnecessary because the unit boundary already is the location.

Labels come from offline execution, used once during data construction and never during training rollouts or inference. We decompose BIRD and Spider training queries into units, sample units from the cold-started model, and execute each sampled unit and its gold counterpart on the training databases. A unit is labeled correct if it returns the same result. Otherwise we assign an error class by comparing its tree against the gold unit. This sidesteps the weakness of rule-based tree matching, which cannot reward semantically equivalent but syntactically different queries.

We add schema-valid hard negatives in two ways. First, we perturb correct units along the error classes ErrorLLM identifies, such as wrong attribute, wrong value, or wrong join key. These are the "silent semantic errors" that execution crashes never reveal. Second, as an optional stage, we let a perturbation generator compete against the gate in the style of Adversarial Training for Process Reward Models. That work trains a step-level reward model against a generator of subtle errors on MATH and SciBench. Grammar constraints bound the space of plausible SQL errors, which should make that game cheaper here than in open-ended math.

Training proceeds in three stages. Supervised fine-tuning teaches the model to emit units followed by verdicts. Reinforcement learning with Group Relative Policy Optimization (GRPO, which normalizes rewards within a group of samples) then uses an outcome reward from final-query execution on the training set, plus a calibration reward on verdicts. At inference, a reject verdict triggers resampling of that unit only, up to a fixed budget. If the budget runs out, the model either abstains or falls back to the best-scored unit.

The verdicts also give a diagnostic dataset that addresses a live disagreement. DAC finds that skeleton parsing is the bottleneck: the clause structure stripped of names. DAC decomposes correction into entity linking and skeleton parsing through repeated prompting of large models. DTS-SQL instead finds that schema linking bounds a 7B model, with a gap of nearly 5% to perfect linking. DTS-SQL fine-tunes separate 7B stages for schema linking and generation. Per-unit error-class predictions, checked against offline labels, tell us which error type actually dominates in a specialized 7B model and which type the gate catches.

Baselines

The naive reference point is Qwen2.5-Coder-7B fine-tuned on the same unit-decomposed data and decoded greedily, with no verdict tokens. A second reference is the same model trained on flat question-to-SQL pairs. These tell us how much decomposition alone buys at this scale.

The strongest existing method is Reward-SQL. It reports 68.9% execution accuracy on BIRD-dev with Qwen2.5-Coder-7B, and 81.7% zero-shot on Spider. We will reproduce it with the same base model, the same training data, and a matched GPU-hour budget for training. We will report results at matched inference cost, measured in generated tokens and database calls per query, because its reranking uses intermediate execution. Beating it means one of two things: matching 68.9% with zero inference-time database calls and fewer tokens per query, or exceeding it at equal tokens. If our reproduction falls short of the reported figure, we will report both numbers and compare against our own reproduction.

The closest decomposition method is component pre-generation with tree-matching rewards. It reports 65.78% on BIRD-dev and 80.7% on Spider-Syn. We will run it on the same base model. It is the exact system whose hallucination chain we target, so we will report how often a wrong unit survives into the final query under each method.

For error detection we compare to ErrorLLM. It reports a detection F1 of 78.12 on BIRD, against below 40 for prompt-based self-correction, and 66.23% execution accuracy with GPT-4o. Its numbers come from a different setting with a frontier generator, so we treat them as a reference rather than a matched comparison. We also reimplement its detector on 7B models applied after full-query generation.

Two ablations isolate the key ingredient. In the first, verdict tokens are trained but resampling is disabled, which separates the value of the training signal from the value of the gate. In the second, the learned gate is replaced by a training-free gate based on agreement among resampled units. We describe this second ablation below.

Why not just…?

Why not just sample many full queries and majority-vote by execution result? This is the most serious threat. It needs no new training, and it attacks hallucination by averaging independent errors. We could find no reported result on Lacuna for execution-based self-consistency with a 7B model on BIRD, so we will run it ourselves. Our argument against it is about compute. Voting pays for a whole query to correct one bad unit. It also cannot fix an error that the model makes systematically, which shows up in most samples. We will plot accuracy against generated tokens for both methods. If voting's curve dominates the gate's, the gate is not worth having and we will say so.

Why not just use agreement among resampled units as the gate, with no training? Do Language Models Know When They're Hallucinating References? shows that inconsistency across independent re-queries exposes fabrication. The same signal could plausibly flag bad units for free. This could work as well as our gate, so it is a full ablation rather than a strawman. We expect it to fail on systematic errors and on units that are short enough that resamples are near-deterministic. If it matches the learned gate, we will adopt it and drop the trained verdict tokens.

Why not just add a separate process reward model, as Reward-SQL does? That works, and it is our strongest baseline. Its costs are a second model and execution of intermediate steps. Its authors also leave open a drift between the policy and the reward model during reinforcement learning. Folding the verdict into the policy removes the second model, and the drift issue becomes a calibration problem that we measure directly. The matched-cost comparison will show whether the extra model was earning its keep.

Why not just apply grammar- and schema-constrained decoding? Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing shows that schema masks drive constraint violations on TOP-style API parsing to 0%, at about 20% latency overhead. That paper also shows structural violations were already below 1%. The remaining errors were pairings that are legal in the schema but wrong in meaning, which masks cannot see. We will apply constrained decoding to every system, so that the gate is credited only for errors that are valid in the schema but wrong in meaning.

Proposed Experiments

The main evaluation uses BIRD-dev execution accuracy and its efficiency score, together with Spider, Spider-Syn, Spider-Realistic, and Spider-DK, which probe robustness to synonyms, missing schema mentions, and required domain knowledge.

Beyond accuracy, we measure four things. Gate quality is detection F1 against offline unit labels. Corruption rate is the fraction of correct units that were flagged and changed into wrong ones. Hallucination-chain rate is the fraction of final queries that contain a known-wrong unit. Cost is measured in tokens and database calls per query.

The decisive check is the accuracy-versus-tokens curve against majority voting and the agreement gate on BIRD-dev, with the same base model and the same sampling temperature. The idea stands only if the learned gate gives higher accuracy at matched tokens and a lower hallucination-chain rate. A second check asks whether the gate transfers. We train the same verdict mechanism on TOPv2 intent-slot trees, where an "unit" is a nested intent. We then test whether it lowers the association-violation rate and raises exact match over constrained decoding alone. Finally, the error-class breakdown reports what fraction of the caught errors are skeleton errors and what fraction are entity errors. That result bears directly on the disagreement between DAC and DTS-SQL about where compact models fail.