LLM4VKG: Leveraging Large Language Models for Virtual Knowledge Graph Construction

Guohui Xiao, Lin Ren, Guilin Qi, Haohan Xue, Marco Di Panfilo, Davide Lanti

Introduction

Virtual Knowledge Graphs (VKGs) provide a powerful paradigm for data integration, allowing users to query relational databases through a "virtual" semantic layer using high-level ontologies and graph query languages like SPARQL. This approach avoids the overhead of physical data migration. However, the manual creation of mappings between database schemas and ontologies remains a significant bottleneck. Traditional rule-based systems often fail due to naming ambiguities and the "fuzzy" semantic nature of database metadata.

The LLM4VKG Framework

To address these challenges, we present LLM4VKG, an automated framework that synergizes structural database analysis with the semantic reasoning capabilities of Large Language Models (LLMs). The construction process is divided into two primary phases:

1. Mapping Pattern Recognition

The framework first abstracts the database schema into a graph representation. It then executes predefined SPARQL queries to identify common structural "Mapping Patterns," such as Schema Entities, Relationships, and Hierarchies. This step provides a formal structural skeleton, ensuring that the final mappings are architecturally sound.

2. Ontology Completion & Mapping Bootstrapping

The core of the semantic matching is handled by a three-stage LLM pipeline:

  • Retriever: Employs a sentence-similarity model (bge-m3) to identify the top-$n$ candidate matches within the ontology for specific database elements.
  • Matcher: A generative LLM (e.g., GPT-4o) evaluates these candidates, assigning a match quality score (High, Medium, or Low).
  • Namer: In cases where no suitable match is found (Low quality), the Namer leverages the LLM's contextual understanding to generate new class or property names, effectively "completing" an incomplete ontology.
The LLM4VKG framework architecture, illustrating the two-phase process of mapping pattern recognition and LLM-driven ontology completion.

Experimental Evaluation

The framework was evaluated using the RODI (Relational-to-Ontology Data Integration) benchmark, which spans diverse domains including Geodata and Oil & Gas. To test robustness, we introduced RODI-T (x%), a variant where varying percentages of the target ontology vocabulary were removed to simulate real-world scenarios of incomplete domain models.

Key Results and Insights

LLM4VKG demonstrates a significant leap over previous state-of-the-art tools like BootOX:

  • Accuracy Gains: LLM4VKG achieved an average F1-score improvement of 17% across all RODI scenarios.
  • Robustness to Naming: In scenarios with obscured or non-standard database naming, the framework showed a 39% absolute improvement in F1-score, highlighting the LLM's superior semantic understanding over lexical rules.
  • Handling Incompleteness: The system proved remarkably resilient to incomplete ontologies. As shown in the performance metrics, LLM4VKG with 25% missing ontology data still performed comparably to prior tools using a complete ontology.
  • Complex Mappings: Unlike traditional tools that struggle with n:1 relationships, LLM4VKG successfully managed complex mapping patterns.
Performance of LLM4VKG (using GPT-4o and GPT-4o-mini) on the RODI-T benchmark with varying levels of ontology incompleteness.

Conclusion

LLM4VKG establishes a new state-of-the-art for automated VKG construction by grounding LLM reasoning within formal structural patterns. Future work will focus on expanding the library of mapping patterns and incorporating data-level statistics to further refine matching accuracy in highly complex database environments.

Summary updated

Abstract

Virtual Knowledge Graphs (VKGs) provide an effective solution for data integration but typically require significant expertise for their construction. This process, involving ontology development, schema analysis, and mapping creation, is often hindered by naming ambiguities and matching issues, which traditional rule-based methods struggle to address. Large language models (LLMs), with their ability to process and generate contextually relevant text, offer a potential solution. In this work, we introduce LLM4VKG, a novel framework that leverages LLMs to automatize VKG construction. Experimental evaluation on the RODI benchmark demonstrates that LLM4VKG surpasses state-of-the-art methods, achieving an average F1-score improvement of +17% and a peak gain of +39%. Moreover, LLM4VKG proves robust against incomplete ontologies and can handle complex mappings where current methods fail.
BibTeX
@inproceedings{xiao2025llm,
  author = {Guohui Xiao and Lin Ren and Guilin Qi and Haohan Xue and Marco Di Panfilo and Davide Lanti},
  title = {{LLM4VKG: Leveraging Large Language Models for Virtual Knowledge Graph Construction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/525},
}
Figure