EvoOntology: A Self-Evolving Ontology Layer for Data Agents

Meiduo Chong, Shaolei Zhang, Ju Fan, Xiaoyong DuarXiv 2026 · arXiv

TL;DR

EvoOntology bridges the "agent-data gap" by introducing an interactive, self-evolving ontology layer served via the Model Context Protocol (MCP). By autonomously building and continuously refining domain knowledge from agent interaction histories, it allows LLM data agents to navigate complex, heterogeneous data sources with significantly higher accuracy and token efficiency.

What Problem Does This Paper Address?

When Large Language Model (LLM) data agents interact with heterogeneous data sources—such as SQL databases, CSV files, and unstructured documents—they face a fundamental agent-data gap. Because the agent does not know the data structure or content a priori, it is forced to blindly issue exploratory queries, guess where specific concepts live, and wade through irrelevant content.

Existing solutions fall short: directly exploring raw data scales poorly and quickly exhausts context windows, while manually creating "semantic layers" to inject into prompts is costly, inflexible, and often distracts the LLM with irrelevant metadata. The authors address the need for a scalable, interactive intermediate layer that explicitly maps domain concepts to physical data and dynamically adapts to specific agent behaviors.

Methods

EvoOntology introduces an agent-first builder-and-evolver framework structured into three distinct layers:

  1. Content Layer: A typed semantic graph containing Terms (domain concepts), Mappings (links to physical data fields), Constraints (business rules), and Evidence (data observations).
  2. Schema Layer: Defines the structural rules and permitted relationship types for the content layer.
  3. Tool Layer: Exposes the ontology to the data agent via Model Context Protocol (MCP) tools, allowing the agent to actively query the ontology at runtime (using fbrowse to search terms and fresolve to retrieve detailed mappings).

The lifecycle of EvoOntology consists of two main phases:

  • Evidence-Grounded Initialization: A builder agent extracts candidate concepts from a training workload and runs executable probes against raw data sources to automatically construct a verified initial ontology.
  • Trajectory-Grounded Evolution: An evolution agent analyzes historical interaction trajectories to diagnose failures. It attributes errors to specific layers, proposes localized patches (edits), and passes them through a Backbone-Conditional Paired Validation Gate to ensure updates genuinely improve performance before deployment.
Figure 2: Overview of EvoOntology. It comprises a typed content graph, its object schema, and a runtime tool interface. The builder constructs an evidence-grounded initial state, while the evolution agent refines it from historical interaction trajectories.

Datasets

The framework was evaluated using four LLM backbones across three diverse data-agent benchmarks:

  • Deep Data Research (DDR-Bench): Evaluated on the 10-K financial scenario, requiring open-ended data research and synthesis across highly heterogeneous sources.
  • InsightBench: A business-analytics benchmark requiring agents to surface insights and summaries from CSV datasets.
  • BIRD: A large-scale text-to-SQL benchmark evaluated under the Oracle Knowledge setting to measure query correctness and execution efficiency.

Key Insights and Ideas

  • On-Demand Tooling Beats Prompt Injection: Dumping an entire semantic layer into an LLM prompt causes instruction distraction and context bloat. Serving the ontology as an interactive MCP tool allows the agent to selectively query only what it needs for the current execution step.
  • Ontology Evolution is LLM-Specific: Different LLM backbones (e.g., GPT vs. Claude) evolve distinct ontologies with relatively low overlap (pairwise Jaccard similarity < 0.62). Cross-backbone transfer experiments show that an ontology evolved for one LLM performs worse when deployed on another, proving that semantic layers must adapt to specific model behaviors.
  • Self-Evolution Enhances Token Efficiency: Rather than increasing computational overhead, a well-evolved ontology shortens the agent's trajectory by reducing repeated schema discovery. This optimization reduces total token consumption per task by approximately 20% compared to a baseline agent without an ontology layer.

Key Results

  • DDR-Bench: EvoOntology achieved a massive average gain of +17.8 points in Trajectory-Wise accuracy over the baseline ReAct agent. In contrast, traditional static semantic layers often degraded performance due to context clutter.
  • BIRD (Text-to-SQL): Delivered consistent improvements across all backbones, yielding an average execution accuracy (EX) boost of 7.4 points and a Valid Efficiency Score (VES) increase of 8.6 points.
  • InsightBench: Improved both insight generation and summary quality, showing a mean overall gain of 1.9 points (and up to +6.1 points on DeepSeek-V4-Flash).
  • Ablation Studies: Confirmed that the validation gate is the most critical element of the evolution loop; removing it caused an 11.2-point drop in accuracy due to unfiltered regressions.

Limitations and Future Work

A dedicated limitations section is not explicitly discussed in the paper. However, the authors' analysis implies a few boundaries:

  • Workload Saturation: The evolution curve flattens out after a few rounds, suggesting the framework stabilizes once it covers the recurrent concepts of a specific workload, but may require re-triggering if the user query distribution shifts drastically.
  • Dependency on Probing: The initialization phase relies heavily on successful execution of database probes; highly messy, unindexed, or restricted raw data sources might hinder the builder agent's initial grounding capability.

Summary updated

Abstract

Data agents aim to fulfill natural-language instructions over heterogeneous data, including tables, files, and databases. However, data agents face a challenging agent-data gap: heterogeneous data resides outside the agent, while the agent can access it (e.g., column names and file paths) only through generic tools. Existing approaches either let agents directly explore raw data sources or inject manually constructed semantic layers into prompts. However, neither scales well to large heterogeneous data sources nor adapts to different agent behaviors. In this paper, we introduce EvoOntology, a self-evolving ontology layer for data agents. EvoOntology encapsulates the ontology as an MCP server comprising a schema layer, a content layer, and a tool layer, enabling agents to actively query and interact with the ontology at runtime. To this end, we introduce a builder agent for autonomous ontology construction and a self-evolution loop that continuously refines the ontology through attribution-guided typed edits that are accepted only after a backbone-conditional paired evaluation. Experiments on three well-adopted data-agent benchmarks with four LLM backbones demonstrate that EvoOntology consistently outperforms strong baselines and existing semantic-layer approaches, effectively bridging the agent-data gap and enabling more effective interaction with heterogeneous data. Code: https://github.com/ruc-datalab/EvoOntology
BibTeX
@misc{chong2026evoontology,
  author = {Meiduo Chong and Shaolei Zhang and Ju Fan and Xiaoyong Du},
  title = {{EvoOntology: A Self-Evolving Ontology Layer for Data Agents}},
  year = {2026},
  eprint = {2609.15779},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.15779},
}
Figure