Analysts who query company databases repeat themselves constantly. They restate that they want metric units, that "active" wells exclude those under maintenance, and that "last quarter" means the fiscal quarter. The tools they use forget all of this between sessions, and often between turns. This proposal stores each such standing preference as a small, typed rule in the vocabulary of a domain ontology, meaning a formal dictionary of business concepts such as :Well, :Measurement and :TemperatureReading. The query compiler then adds a remembered rule to a new query only when that query touches the concept the rule is about. A new instruction overrides a remembered rule by deleting or replacing one typed object, so the system never has to rewrite SQL text by guesswork.
The substrate is ontology-based data access (OBDA). In this approach a user poses a query over ontology concepts, and declarative mappings translate ("unfold") it into SQL that runs on the original databases, with no copying of data (Optique; Kharlamov et al.). Unfolding today is stateless: every query is compiled as if the user had never been seen before. The central bet is that the ontology, built to integrate data sources, is also the right index for user memory. Concept-level scoping should let a preference stated about one database carry over to another database mapped to the same concept. It should keep memories about unrelated topics out of the current query. It should also turn "forget that filter" into a well-defined operation.
Core Claim
Long-term user memory for structured data querying should be kept as defeasible constraints, meaning defaults that a later statement can override. Each constraint is attached to an ontology concept, not stored as free text or embeddings. Concretely, a preference is $p = (C, \varphi, s, t)$. Here $C$ is an ontology concept, $\varphi$ is a constraint written in ontology vocabulary (a filter, unit convention, default time window, or projection), $s$ is a scope (global, or tied to a project), and $t$ is the time the preference was stated. Given a user query $q$ over ontology terms, the compiler unfolds
$$q^{*} = q \wedge \bigwedge_{p_i \in P(q)} \varphi_i, \qquad P(q) = \{\, p_i : C_i \sqsupseteq \text{some concept used in } q,\ p_i \text{ not overridden by } q \,\}.$$
In this expression $C_i \sqsupseteq$ means "equals or is a superclass of", as decided by the ontology's subclass axioms. "Overridden" means $q$ or the current dialogue states a constraint on the same concept and attribute that conflicts with $\varphi_i$. An LLM is used only to translate each utterance into ontology-level operations: add, retract, or replace a constraint, or pose a query. Everything after that step is symbolic.
What would be new is the claim that three separately documented failures share one fix. The first is the interaction tax of stateless agents (Tell Me More!). The second is context noise when history is applied after a topic switch (MM6 at NTCIR-16). The third is the gap between recognizing a constraint change and correctly resolving it (TIDE-BENCH). Concept-scoped typed memory is proposed as that shared fix.
Where This Sits
Several strands of work motivate the proposal. LostNet couples within-session attention with an external memory of past sessions for web reranking, and shows that removing long-term memory hurts. Its memory, however, is a set of dense slots over unstructured logs, with no schema and no executable query. Mem-PAL organizes months of user history into logs, topic outlines, profiles and clustered "principles". Those principles are text and are never tied to a database. LongMemEval and MemoryArena show that long-context models and commercial memory systems degrade badly as sessions accumulate. We reuse their problem categories, especially knowledge updates and abstention, but move them onto executable data queries.
Other work constrains the design. TIDE-BENCH reports that models recognize intent drift more than 94% of the time but resolve it correctly only about 45–55% of the time. It also finds constraint-level drift harder than projection changes, which is why retraction operates on typed objects rather than on SQL strings. MM6 shows that applying session history blindly can do worse than stateless querying. Any design that injects memory unconditionally is therefore ruled out, and concept scoping is our answer.
Two papers supply methods. Satyrn shows that a light semantic layer plus a structured intermediate representation compiled deterministically to SQL raises factual accuracy to 86%, against 57% for GPT-4 with Code Interpreter. We borrow the principle that the LLM should emit a structured plan, not SQL. Satyrn is single-turn and stateless; we add persistent state to its kind of intermediate representation. LLM4VKG bootstraps ontology-to-schema mappings and is evaluated on RODI. We use it to build mappings for databases that lack them, and we treat its mapping errors as a confound to be measured. BIRD-INTERACT supplies the function-driven user simulator design (reported 97.3% accuracy on its UNA questions) that we adapt to multi-session users.
Research Program
The first component is the system. It keeps a per-user preference store of $(C,\varphi,s,t)$ records, validated against the ontology so that a constraint on a nonexistent attribute is rejected at write time. An utterance parser, an LLM prompted with the ontology signature and the user's active preferences, emits an operation sequence. A scoping and override resolver computes $P(q)$ using subclass reasoning, and a standard OBDA unfolder produces SQL. When two stored preferences conflict, the newer one wins within the same scope and the project scope beats the global scope. The resolver is also allowed to ask the user rather than guess: if a preference is older than a threshold and the current query is on its concept, it asks one confirmation question.
The second component is a benchmark, because no standardized test combines persistent users with database querying (the file's Gap 2). We call it MS-OBDA. It is built on the RODI scenarios, which come with ontologies and gold mappings, and on a subset of BIRD databases with LLM4VKG-bootstrapped, hand-corrected mappings. Each simulated user has a latent preference profile and works through roughly 10–30 sessions over weeks of simulated time. In the style of TIDE-BENCH, we inject controlled events: a preference is stated, applied silently, retracted, replaced, scoped to a project, or made irrelevant by a topic switch.
The paired factorial construction borrowed from TIDE-BENCH lets us attribute each failure to a specific event type. Some profiles include cross-source queries, where a preference stated while querying database A must apply to database B through a shared concept. The simulator answers only from the latent profile through fixed functions, which limits the answer leakage the file flags as a risk for simulators.
The metrics are defined at the level of executed queries. Session execution accuracy compares result sets with gold. Preference violation rate counts active preferences that were wrongly omitted. Over-application rate counts preferences applied where they should not be, which is our direct measure of context interference. Retraction accuracy is scored on retract and replace events. The interaction tax is the number of clarification questions about facts the user already stated. Token and latency cost are reported per session.
Baselines
The naive reference is a stateless OBDA or text-to-SQL agent that remembers nothing across sessions but may ask clarifying questions within a session. It shows the ceiling on interaction tax and the floor on preference compliance. It uses the same backbone LLM and the same simulator.
The strongest existing competitor is the best LLM text-to-SQL agent given the user's full multi-session history in context, as in the long-context setting of LongMemEval, with the database schema and the ontology-derived documentation also in the prompt. On single-turn messy databases the best setup in BIRD reached 55.90% execution accuracy, and conversationally Claude-Sonnet-4.6 reached only 27.8% dialogue execution accuracy on TIDE-BENCH joint samples. No reported result was found on Lacuna for any system on multi-session, preference-bearing database querying, so this baseline's numbers on MS-OBDA will be produced by us.
The second strong competitor is retrieval over text memory. We implement it as LongMemEval's Chain-of-Note with JSON-structured history (reported gain of up to 10 absolute points) and as an H²Memory-style profile plus principle store from Mem-PAL, both feeding the same LLM SQL generator.
Fairness is enforced in the same way for every system. All systems share one backbone LLM, one prompt-tuning budget (a fixed number of development sessions), identical retrieval token budgets for the memory baselines, and one simulator seed set. Beating a baseline means a statistically significant gain in session execution accuracy of at least 5 points over the best competitor. Preference violation and over-application must not rise at the same time; a gain bought by over-applying memory does not count.
Three ablations isolate the key ingredient. The first keeps typed constraints but attaches them to raw schema columns instead of ontology concepts. It removes the ontology while keeping the structure. The second keeps ontology typing but applies every stored preference regardless of concept, which removes scoping. The third drops the override logic and lets the LLM rewrite the query from the preference list, which removes symbolic retraction.
Why not just…?
Why not just put the whole history into a long-context model? LongMemEval reports a 30–60% decline for long-context LLMs relative to seeing only the relevant evidence, and MemoryArena reports near-zero success in its constraint-heavy group travel domain. That evidence comes from chat and agent tasks, not SQL, so the long-context baseline is run directly on MS-OBDA. The comparison is made at matched session counts, and we plot accuracy against the number of accumulated sessions. The proposal predicts a widening gap. If the curves stay parallel at realistic history lengths, the memory design is not justified on accuracy, and only the cost argument remains.
Why not just retrieve past text notes and let the LLM write SQL? This is the most serious threat, because retrieval handles the scale problem. The evidence against it is indirect. TIDE-BENCH shows that recognizing a changed constraint is far easier than applying the change (over 40 points apart), and text memory leaves the application step to the LLM. The retraction and replace events in MS-OBDA test exactly this. If retrieval with Chain-of-Note matches our retraction accuracy and over-application rate, typed memory is unnecessary and we will say so.
Why not just type the preferences against database columns and skip the ontology? Within a single database this may work equally well, and the first ablation measures that honestly. The ontology should matter in two places. One is cross-source queries, where a column-bound preference cannot reach a second database. The other is subclass generalization, where a preference on :Sensor should also cover :TemperatureSensor. If the column-typed ablation matches the full system on the cross-source split as well, the ontology contributes nothing beyond engineering convenience, and the claim shrinks to "typed memory helps".
Why not just ask the user every time? Asking is always correct, but it is what users complain about. The naive baseline measures this directly by reporting the accuracy it buys and the number of questions it spends. Our system must reach comparable accuracy with far fewer questions about already-stated facts.
Proposed Experiments
The decisive check for the ontology is the cross-source split of MS-OBDA. The full system should beat the column-typed ablation by a clear margin there and roughly tie it within a single source. If that pattern does not appear, the ontology-scoping hypothesis is wrong.
The decisive check for scoping is the topic-switch split, which reproduces the MM6 interference condition inside a database setting. The full system's over-application rate should sit near the stateless baseline's, and the scoping-removed ablation and long-context baseline should be clearly worse.
The decisive check for symbolic override has two parts. The first uses the retract and replace events of MS-OBDA. The second reruns the drift-only portion of TIDE-BENCH on BIRD schemas, with LLM4VKG-bootstrapped ontologies and a mapping-correctness audit reported alongside. There the target is to close part of the recognition–resolution gap, measured against the reported ~45–55% drift resolution accuracy. Because our ontologies for BIRD are bootstrapped, we also report results restricted to queries whose mappings were verified correct, to separate mapping errors from memory errors.
Finally, a small study with 10–15 people using RODI's Oil & Gas scenario checks whether the simulator's preference events resemble what people actually state and retract. If they do not, the benchmark's event distribution is reweighted before any headline claim is made.
References
- Optique: Ontology-Based Data Access Platform
- Semantic Rule-Based Equipment Diagnostic
- Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents
- MM6 at the NTCIR-16 Session Search Task
- Evaluating LLMs on Conversational Text-to-SQL under Chain Ambiguity and Intent Drift
- Long Short-Term Session Search: Joint Personalized Reranking and Next Query Prediction
- Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interaction
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
- Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
- Satyrn: A Platform for Analytics Augmented Generation
- LLM4VKG: Leveraging Large Language Models for Virtual Knowledge Graph Construction
- BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation via Lens of Dynamic Interactions
- Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs