Let the Language Model Name Concepts and Let the Reasoner Unfold Them: Ontology Depth as the Hidden Cause of Distraction in Natural-Language Database Querying

Ask a database "which suppliers ship parts that some certified lab has tested?" and two separate problems arise. The first is a vocabulary gap: the user says "supplier" and "certified lab", while the tables say vendor_id and org_type = 3. The second is hidden structure. A formal ontology (a set of logical rules stating, for example, "every tested part has some test report, and every test report is issued by some lab") lets the database answer questions about things that are only implied, not stored. A common way to close the vocabulary gap with a large language model is to put the ontology's labels, synonyms and rules into the prompt and ask the model to write SQL over the raw tables. We argue that this quietly turns a logic problem into a distraction problem. The amount of context the model must hold in mind to write a correct query grows with the ontology's depth, meaning how many layers of implied, unnamed objects a query can pass through. Theory says this growth is polynomial at depth 1 and exponential from depth 2. Language models are known to degrade as irrelevant or merely voluminous context piles up.

The proposal is to split the job. The language model handles only the vocabulary gap: it translates the user's words into a query written in the ontology's own concept and relation names, and it sees only those names and their labels. A symbolic query rewriter then does the depth-dependent unfolding into SQL, which the model never sees. We predict, and design experiments to test, that prompt-based unfolding loses accuracy sharply when queries require reasoning through depth-2 structure, while the split design stays flat in depth. Its remaining error should be explained by how many ontology terms it has to choose among, not by how deep the ontology is.

Core Claim

Work on ontology-based data access (answering queries over a database through an ontology) shows that the size of the rewritten query is driven by structural depth, not by the number of axioms. Query Rewriting over Shallow Ontologies formalizes rewriting through tree-witnesses: minimal pieces of a query that can be satisfied by anonymous objects the ontology guarantees to exist. That paper shows depth-1 ontologies admit polynomial-size nonrecursive Datalog rewritings. Depth 2 already encodes NP-hard problems and forces exponential blowup even for simple linear queries (Succinctness of Query Rewriting in OWL 2 QL: The Case of Tree-like Queries; Circuit Complexity Meets Ontology-Based Data Access).

Our claim is that this result has a direct empirical counterpart for language models. If a model is asked to write source-level SQL, it must implicitly enumerate the same tree-witness combinations. The relevant context it must attend to then scales like the rewriting does, and distraction grows with it.

This splits the difficulty of natural-language querying over ontologies into two separable costs. The first is a lexical cost, which scales with the size of the ontology's vocabulary. The second is an unfolding cost, which scales with depth. Current systems pay both inside the prompt. The new element is the measurement itself: how a language model's accuracy depends on ontology depth and tree-witness count, which no paper in this literature reports. A second new element is a design whose prompt grows only with the first cost.

Relation to Prior Work

Three lines of work motivate the problem without addressing depth.

Text-to-SQL systems treat the schema as a flat set of items to link. RAT-SQL encodes question–schema links as graph relations and shows schema linking matters: accuracy fell from about 60% to 40% without it. RESDSQL ranks schema items and hard-filters to the top few. Any false negative is unrecoverable. Solid-SQL keeps the full schema in the prompt and adds "focus on" hints to avoid that failure. All three operate on stored tables with single-hop foreign keys. None models implied objects or how context requirements grow with inference depth.

On the noise side, OG-RAG uses an ontology to index documents as a hypergraph and selects a minimal covering set of fact groups. This is the closest design in spirit, but it prunes stored facts and performs no deductive rewriting. Superposition Prompting and DAS reduce distraction across retrieved documents. They show that removing context helps, which motivates our approach. They also expose the false-negative risk of pruning multi-hop evidence, which constrains us: our design must not prune what a deep query needs.

On the vocabulary side, ECLASS-Augmented Semantic Product Search supplies a key cautionary finding. Adding hierarchical ECLASS metadata helped retrieval, but adding full "Advanced" attribute metadata hurt it, and LLM query rewriting stripped technical detail. Richer ontology context is therefore not free, which is exactly the tension our decomposition explains. RaFe and InteR rewrite queries against unstructured indexes. InteR's degradation beyond two rounds is a further sign that extra generated context turns into noise.

The method is borrowed from the theory line: tree-witness rewriting from Query Rewriting over Shallow Ontologies, with depth bounds that keep tree-shaped queries tractable (Combined Complexity of Answering Tree-like Queries in OWL 2 QL). That paper also names the open question we partly address: how often worst-case blowups occur in practice.

Research Program

We would build three things.

The first is a depth-controlled benchmark, which we call Spider-OBDA. For a subset of Spider databases, we write OWL 2 QL ontologies (the lightweight ontology language used in these theory papers) together with mappings from ontology terms to tables. The ontologies are built in families that differ only in depth: 0 (plain renaming and subclass rules), 1, 2, 3, and cyclic. Concept labels are perturbed in the style of Spider-Syn and Spider-Realistic, so the vocabulary gap is present at every depth level. Questions are written against the ontology, partly by re-expressing Spider questions and partly by new authoring targeted at queries that do and do not need tree-witnesses. Gold answers are certain answers computed by the symbolic rewriter rather than hand-written SQL. Each question is tagged with its depth and its tree-witness count.

The second is the split system. A language model receives the question plus the ontology's concept and relation names with their labels and synonyms. It produces a conjunctive query (a single join of conditions, the query class the theory covers) over ontology terms. A tree-witness rewriter then compiles that query into SQL over the source tables. Optionally, the model also sees the immediate subclass neighborhood of candidate terms, so it can choose the right level of generality.

The third is an analysis relating every system's errors to two predictors: the number of ontology terms in the prompt, as a lexical-noise proxy, and the query's tree-witness count, as an unfolding proxy. On the ECLASS taxonomy we would also check whether a realistic industrial hierarchy actually produces depth-2 tree-witnesses, a small empirical answer to the worst-case-versus-practice question.

Baselines

The naive reference point is a language model given the source schema only, with no ontology, writing SQL directly. It shows how much the ontology helps at all.

The strongest existing method for this question is Solid-SQL-style prompting (Solid-SQL). The model receives the full source schema plus the full ontology rules and mappings, with "focus on" hints for linked items. It is the strongest competitor because it was designed for robustness to vocabulary perturbation without the false negatives of hard filtering. It reports 82.1% execution accuracy on Spider, 58.9% on BIRD, and an average 11.6% improvement over baselines on Spider-Syn, Spider-Realistic and Dr. Spider. On Spider-OBDA there is no reported result found on Lacuna, so we reimplement it with the same base model as our system and the same number of in-context examples and generation rounds.

A hard-filtering baseline follows RESDSQL (84.1% execution accuracy on Spider dev with NatSQL; no reported Spider-Syn number found on Lacuna for RESDSQL). Its ranker is retrained on Spider-OBDA to pick top-k tables and rules. An OG-RAG-style baseline (OG-RAG) greedily selects a minimal set of ontology axioms that covers the question's linked concepts.

Two ablations isolate the key ingredient. The first gives the split system's language model the full rules as well as the vocabulary, which tests whether hiding the unfolding is what matters. The second truncates the symbolic rewriter at depth 1, which tests whether the deep unfolding is actually needed.

All systems share the same base model, decoding budget and tuning budget, matched as equal numbers of development-set prompt or hyperparameter trials. The trained ranker gets the same training split. Beating the strongest baseline means two things. Overall execution accuracy must be no lower. And the slope of accuracy against tree-witness count must be significantly flatter, with a clear gap on depth-2 and depth-3 questions. A win that comes only from depth-0 questions would not count.

Why not just…?

Why not just give a long-context model everything and let it reason? This is the Solid-SQL baseline taken to its limit. The distraction literature predicts it degrades as relevant-but-voluminous context grows: DAS reports large losses from naive readers under 100 retrieved documents, and ECLASS found richer metadata hurt retrieval. At depth 0 and 1 the required unfolding is small and this baseline may well match us. The experiment settles the question by reporting accuracy separately per depth level.

Why not just materialize the ontology into the database and run ordinary text-to-SQL? This is a real threat. Materializing each concept as a view lets the model see a flat schema in ontology vocabulary, which is nearly our interface. Per-concept views cannot capture joins that pass through anonymous objects. So we predict this baseline matches the split system on tree-witness-free questions and falls behind as tree-witness count rises. Full chase materialization, which adds the implied objects as placeholder rows, would actually match our accuracy on finite-depth ontologies, because the model's task is then identical. In that case we would state plainly that the contribution is the interface, not the backend. Chase materialization changes the data and does not terminate on cyclic ontologies, and we include it so the comparison is explicit.

Why not just fine-tune a parser on depth-varied data? A trained parser might learn common unfolding patterns. But the number of distinct source-level SQL shapes grows with the rewriting size, which is exponential from depth 2, so required training data should grow with depth. We train a RESDSQL-style parser on matched data budgets and plot accuracy against depth. A flat curve would refute our explanation.

Proposed Experiments and Decisive Checks

The main experiment runs all systems on Spider-OBDA and reports execution accuracy against certain answers. Results are broken down by depth, by tree-witness count, and by vocabulary perturbation type. Prompt length and latency are reported alongside accuracy.

The first decisive check is a regression of each system's correctness on prompt vocabulary size and tree-witness count. Our account predicts that tree-witness count is the dominant predictor for all prompt-unfolding baselines and near-zero for the split system. For the split system, vocabulary size should explain most of the remaining error, which matches the depth-0 behavior of the ECLASS metadata finding. If the split system's accuracy also falls with depth, the explanation fails, because its prompt does not grow with depth.

The second check is a stress test on the depth-2 clique-style construction used in the succinctness proofs, padded with realistic labels. Here the rewriting provably explodes. Prompt-based systems should approach the near-zero Extra-Hard behavior Spider reports for unseen databases (0.3% for early models), while the split system's accuracy should hold and its cost should move into the database.

The third check is a count of tree-witness-bearing queries in the ECLASS-derived ontology. It tells us whether the depth effect is a practical concern or mainly a theoretical one. Either outcome is worth reporting.