RubikSQL: Lifelong Learning Agentic Knowledge Base as an Industrial NL2SQL System

Zui Chen, Han Li, Xinhao Zhang, Xiaoyu Chen, Chunyin Dong, Yifeng Wang, Xin Cai, Su Zhang, Ziqi Li, Chi Ding, Jinxu Li, Shuai Wang, Dousheng Zhao, Sanhai Gao, Guangyi LiuarXiv 2025 · DOI · arXiv

Abstract

RubikSQL is an industrial-grade NL2SQL (Natural Language to SQL) system that treats query generation as a lifelong learning task rather than a one-off translation. By maintaining an "Agentic Knowledge Base" that evolves with user interactions, it successfully tackles the "messy" realities of enterprise data—such as implicit user intents, domain-specific jargon, and massive, complex schemas—achieving state-of-the-art performance on major benchmarks.

The Industrial NL2SQL Challenge

While Large Language Models (LLMs) have revolutionized NL2SQL, a massive gap remains between "benchmark performance" and "industrial utility." In a real-world enterprise, queries are rarely textbook-perfect. RubikSQL identifies four critical hurdles:

  1. Implicit Intent: Users often expect complex logic (e.g., Year-over-Year comparisons) not explicitly stated in the query.
  2. Private Domain Knowledge: Companies use non-standard acronyms and custom financial formulas.
  3. Wide Table Schemas: Databases often contain hundreds of columns requiring complex joins.
  4. Context Sensitivity: Query meanings change based on data update cycles or user roles.

Unlike benchmarks like Spider that focus on cross-database generalization, RubikSQL focuses on specialization—becoming an expert on a single, complex enterprise ecosystem over time.

Methodology: The Agentic Knowledge Base

The core of RubikSQL is its Agentic Knowledge Base (KB), which functions as a semantic layer decoupling knowledge storage from its utilization. This system is powered by a new standardized format called UKF 1.0 (Unified Knowledge Format).

The RubikSQL end-to-end pipeline, illustrating the lifelong learning loop from context engineering to knowledge distillation.

System Architecture

As shown in the architecture diagram, RubikSQL operates across four distinct layers: Storage, Semantic, Engine, and App. The Unified Knowledge Format (UKF) acts as the critical bridge, ensuring that domain knowledge is accessible regardless of the underlying storage medium (e.g., relational databases, vector stores, or file systems).

The layered architecture of RubikSQL, highlighting the Unified Knowledge Format (UKF) as the semantic bridge between storage and the multi-agent engine.

Multi-Agent Workflow

The system employs three specialized agents to handle the generation process:

  • RAG Agent: Retrieves relevant domain knowledge and schema snippets.
  • SQL Gen Agent: Drafts the SQL using Chain-of-Thought (CoT) reasoning.
  • SQL Refine Agent: Executes the code in a sandbox, catches errors, and iterates until the query is valid.

Lifelong Learning and Knowledge Distillation

RubikSQL treats the query stream as a source of truth. It mines both labeled and unlabeled user queries to extract "experiences" (CoTs, subqueries, and comments), which are fed back into the KB. To maintain speed and cost-efficiency, a "teacher" model (like DeepSeek-V3/R1) generates high-quality reasoning traces to fine-tune smaller "student" models (like Qwen-32B).

Evaluation and Results

RubikSQL was evaluated on several benchmarks, including the new RubikBench, a large-scale dataset derived from a real multinational car manufacturer's financial database featuring ~5,000 complex queries.

  • State-of-the-Art Performance: RubikSQL achieved 77.3% Execution Accuracy (EX) on the BIRD Mini-Dev benchmark, surpassing previous leaders.
  • KaggleDBQA: It reached 58.9% EX, a significant jump over existing baselines.
  • Efficiency: Through knowledge distillation, a fine-tuned 32B model rivaled the performance of much larger teacher models when equipped with the RubikSQL knowledge base.

Key Insights and Future Work

A standout finding of this research is that only 12% of errors were caused by insufficient LLM reasoning. The vast majority of failures stemmed from missing domain knowledge or misunderstood user intent. This suggests that the primary bottleneck for industrial NL2SQL is knowledge management, not model size.

Future work will focus on deeper user profiling and more sophisticated autonomous search capabilities within the knowledge base to handle increasingly ambiguous requests.

Summary updated

BibTeX
@misc{chen2025rubiksql,
  author = {Zui Chen and Han Li and Xinhao Zhang and Xiaoyu Chen and Chunyin Dong and Yifeng Wang and Xin Cai and Su Zhang and Ziqi Li and Chi Ding and Jinxu Li and Shuai Wang and Dousheng Zhao and Sanhai Gao and Guangyi Liu},
  title = {{RubikSQL: Lifelong Learning Agentic Knowledge Base as an Industrial NL2SQL System}},
  year = {2025},
  eprint = {2508.17590},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.17590},
}
Figure