LuthorSQL: A One-Pass Text-to-SQL Model as System 1 for a Slower LLM Agent

Ibrahim Memon, Armin Karamzade, M. Hanif Jhaveri, Roy FoxarXiv 2026 · arXiv

TL;DR

LuthorSQL is a fast, local 7B model that acts as a "System 1" gate for Text-to-SQL tasks, predicting query plans in a single forward pass. By handling routine queries locally and escalating only uncertain cases to frontier models like Claude 3.5 Opus, it maintains state-of-the-art accuracy while slashing API costs by up to 34%.

What Problem Does This Paper Address?

Frontier LLM agents (like Claude 3.5 Opus or GPT-4o) are currently the gold standard for translating natural language into SQL. However, they come with three major baggage items: they are slow, expensive (costing tens of thousands of dollars annually for high-volume enterprise use), and privacy-leaking (sending sensitive schema and query data to external APIs).

The authors ask: To what extent can we offload this work to local hardware without losing accuracy? They draw inspiration from Daniel Kahneman’s "System 1 and System 2" framework, designing a fast, intuitive "System 1" (LuthorSQL) to handle the easy stuff and a slow, deliberate "System 2" (a frontier LLM agent) for the hard stuff.

Methods

LuthorSQL moves away from standard token-by-token autoregressive generation. Instead, it uses a non-autoregressive slot-filling approach:

  1. Structured Plans: The model (a 7B OmniSQL backbone with LoRA adapters) makes a single forward pass to fill "slots" in a bounded query grammar (e.g., projections, filters, joins). This prevents syntax errors and hallucinations.
  2. Deterministic Compiler: Instead of the model writing complex JOIN clauses, a compiler takes the predicted plan and finds the shortest path between tables using a Steiner tree over the database's foreign-key graph.
  3. Candidate Enumeration & Validation: The model generates up to 32 candidate plans. A second adapter, VALHARD, scores these candidates after they are executed against the database.
  4. Confidence Cascade: If the margin between the top two candidates is high, LuthorSQL provides the answer. If the margin is low (indicating uncertainty), the query is escalated to the "System 2" LLM agent.
Figure 1: How LuthorSQL answers a question. The slot heads, read after the question, point to schema columns and to a stored value; the word links are drawn for illustration, since the model scores slot states, not word alignments. No question word names characters; the plan lists it only in its require slot, and the compiler builds the join along the foreign-key path between movie and actor. The same pass yields up to 32 distinct plans (21 for this question), found by best-first search, which are compiled, executed and scored by the validator. Example: BIRD train question 740 (movie, a held-out database), answered correctly.

Datasets

The authors utilized several major benchmarks for training and evaluation:

  • Training: BIRD, Spider, and SynSQL-2.5M.
  • Evaluation: BIRD (dev and hidden test sets), KaggleDBQA, and Spider-Realistic.
  • Fresh Benchmark: Crucially, the authors built a new set of 227 questions on databases published after the training cutoffs of major LLMs (July–October 2026) to ensure results weren't just the product of memorization.

Key Insights and Ideas

  • One-Pass Efficiency: By predicting all slots in parallel, LuthorSQL achieves a median latency of just 0.84s, significantly faster than autoregressive models that generate SQL token-by-token.
  • Zero-Shot Dialect Transfer: Because the model predicts an engine-agnostic plan rather than raw SQL string, it can be transpiled to Snowflake, BigQuery, MySQL, etc., with a performance drop of only ~3 points compared to its native SQLite accuracy.
  • The Memorization Trap: Contamination probing revealed that Claude Opus could reproduce 48–92% of public benchmark gold queries verbatim without even seeing the schema. This underscores the importance of the authors' "Fresh" post-cutoff evaluation.
  • Tool-Use Overcomes Recall: Despite memorizing benchmarks, the study found that LLM agents maintain high accuracy on renamed schemas and new data because they use database tools to explore, rather than relying solely on internal weights.

Key Results

  • Standalone Performance: LuthorSQL reached 68.03% execution accuracy on the BIRD hidden test set. This matches the performance of much larger models (like the 32B Arctic-ExCoT) while using a fraction of the parameters.
  • Cost Savings: In a pre-registered confidence cascade, the system handled roughly half of all queries locally, reducing API costs by 33.3% on the "Fresh" dataset at equal accuracy to the standalone frontier model.
  • Hardware Efficiency: The model runs in just 9 GB of memory with 8-bit quantization, making it viable for deployment on consumer-grade hardware like an Apple M3 laptop.
Figure 2: Cost saving of the t = 3.0 cascade in the held- out tests against Opus alone (B0, circles) and Opus with System 1’s candidates on every question (B1, squares), with System 1 on local hardware. RES: RESERVED (348 BIRD questions; pool: three runs); External: Kag- gleDBQA and Spider-Realistic (693); Fresh: 227 post- cutoff questions. Dashed line: 30%.

Limitations and Future Work

  • Grammar Coverage: The current bounded grammar covers about 84.6% of BIRD dev queries. It struggles with complex features like Common Table Expressions (CTEs), window functions, or deeply nested subqueries.
  • Validator Tuning: The VALHARD validator was tuned on specific splits, which might lead to slightly optimistic results on those specific distributions.
  • Future Directions: The authors suggest developing a dedicated "out-of-grammar" detector to route complex queries to System 2 even before System 1 attempts them, and moving toward native generation in multiple SQL dialects.

Summary updated

Abstract

Frontier LLM agents that query databases with tools are among the most accurate text-to-SQL systems, but per-query calls to a hosted model are slow, costly, and expose sensitive organizational queries to external APIs. For example, a Claude Opus 5.5 agent serving 10,000 queries a day costs over $70,000 a year at list price. To what extent can local hardware already owned by an organization offload this burden without compromising the agent's accuracy? We present LuthorSQL, a non-autoregressive 7B text-to-SQL model that serves as a rapid System 1 gate for a slower LLM agent (System 2), escalating queries only when uncertain. LuthorSQL makes a single forward pass to predict slot scores over a bounded grammar for query plans, compiles and runs up to 32 candidate plans, and chooses an answer using a trained validator. On its own, LuthorSQL reaches 68.03% execution accuracy on BIRD's hidden test set at 0.84 s per question on a single GPU, maintains its accuracy in 9 GB with 8-bit quantized weights, and transfers zero-shot to Snowflake, BigQuery, MySQL, PostgreSQL, and DuckDB within 3.1 points of its native SQLite accuracy. Combined with Claude Opus 5.5 in a pre-registered confidence cascade over 227 questions on databases published after Opus's training cutoff, LuthorSQL answers nearly half of all questions locally and cuts API cost by 33.3% at the same accuracy. Finally, contamination probing shows that Opus reproduces 48-92% of public benchmark gold queries verbatim without seeing the schema; however, its tool-assisted accuracy holds on renamed schemas and reaches 95.6-97.4% on post-cutoff databases. Code and data: https://github.com/IAMIbrahimmemon/luthorsql
BibTeX
@misc{memon2026luthorsql,
  author = {Ibrahim Memon and Armin Karamzade and M. Hanif Jhaveri and Roy Fox},
  title = {{LuthorSQL: A One-Pass Text-to-SQL Model as System 1 for a Slower LLM Agent}},
  year = {2026},
  eprint = {2611.17780},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17780},
}
Figure