TL;DR
The Living Context Graph (LCG) is a continually learned semantic layer that bridges the gap between raw database schemas and actual business logic. By bootstrapping metadata and learning reusable business rules from verified query outcomes, LCG improves text-to-SQL accuracy on unseen questions by over 22 percentage points while significantly reducing inference latency and cost.
What Problem Does This Paper Address?
In enterprise environments, the "data foundation" is often the bottleneck for AI agents. While modern LLMs are excellent at reasoning, they lack the "tribal knowledge" required to navigate real-world databases. A column named net_sales_amt rarely explains which exclusions apply or how fiscal periods are defined; that context is usually buried in old analyst queries or the heads of veteran employees.
Traditional solutions—like manually authoring a semantic layer or building a massive ontology—are expensive, brittle, and quickly become stale. The authors propose a system that learns this context automatically and incrementally through use, without requiring a copy or transformation of the underlying data warehouse.
Methods
The LCG architecture consists of four primary components (C1–C4) that transform a raw warehouse into an AI-ready reasoning engine:
- C1: Bootstrapping: The system performs an initial profiling pass to extract metadata and value sets, creating a "best guess" semantic layer.
- C2: Concept Graph: A dynamic index that maps user vocabulary to the semantic store. It routes queries to the relevant context and evolves as users interact with the system.
- C3: Semantic Learning: When a query outcome is verified (e.g., via a correct SQL execution), the system diagnoses what context was missing and writes a "semantic delta"—a durable, reusable rule like a metric definition or a mandatory filter.
- C4: Validation and Forgetting: This acts as a control loop. An "admission gate" (often an LLM judge) decides if a new rule is trustworthy before it's allowed to serve. Conversely, if a previously learned route leads to a failure, the system can withdraw or "forget" that context.
Datasets
The authors evaluated LCG using a transfer-aware split of three databases from the BIRD (Big Bench for Large-scale Database Grounded Text-to-SQLs) development set:
card_gamestoxicologythrombosis_prediction
The split is designed to test transfer, not just memorization. They ensured that for every "held-out" question, the specific business concepts required to answer it were present in the "learning history" questions, but the questions themselves were distinct.
Key Insights and Ideas
- Semantics Drive Accuracy, Graphs Drive Efficiency: The semantic store (the rules) is what actually makes the agent smarter. The graph structure (the routing) is what makes it fast. Ablating the graph barely changed accuracy but caused a 2.3–3.3x spike in query costs.
- Outcome-Gated Admission is Critical: You cannot simply learn every rule proposed by an agent. Without a "judge" to filter learned rules, accuracy on held-out sets plummeted from 36.3% to 19.9%. Validation is a first-class citizen in continual learning.
- Learning Beats Retrieval: LCG outperformed a five-shot retrieval baseline (RAG) by +12.3 points. This suggests that distilling experiences into durable semantic rules is more effective than simply searching for similar past examples.
- No-Copy Architecture: LCG leaves the underlying enterprise data in place, acting as a lightweight, inspectable metadata layer rather than a new data silo.
Key Results
The quantitative gains were substantial across the board:
- Accuracy Boost: Learning from the history split raised accuracy on the untouched held-out transfer set by +22.6 percentage points (from 13.7% to 36.3%).
- Efficiency Gains: The concept graph reduced median latency by 1.8–3.4x compared to a system without graph routing.
- Incremental Parity: The system reached the same accuracy endpoint whether it learned from a batch of history or through incremental, one-shot interactions, proving its viability for live deployments.
Limitations and Future Work
- Supervision Strength: The current learning loop relies on "verified correct SQL" via answer matching. In real-world deployments, "no-complaint" feedback from users is a noisier signal that the authors haven't yet fully explored.
- Human-in-the-Loop: While the system is highly automated, the authors acknowledge that the "write path" (admitting new rules) still benefits from human review to ensure long-term sync with business goals.
- Scale: The evaluation was limited to three databases (499 questions). Scaling this to massive, multi-thousand-table enterprise schemas remains a future challenge.
- History Saturation: Accuracy tends to plateau once the system has "seen" all the core business concepts in a domain, suggesting a limit to how much can be learned from a finite history.
Summary updated