TL;DR
This paper introduces RIGOR, a novel LLM-driven pipeline that automatically generates rich OWL ontologies from relational databases with minimal human effort. By leveraging retrieval-augmented generation (RAG) and an iterative refinement process, RIGOR significantly improves ontology quality and reduces manual labor compared to existing methods.
What Problem Does This Paper Address?
Relational databases are ubiquitous for data management, but accessing their content semantically and integrating them with other knowledge sources is challenging. Transforming these databases into knowledge graphs (KGs) with well-defined ontologies can unlock advanced data sharing, learning, and reasoning capabilities. However, previous methods for ontology extraction from relational databases either require substantial manual effort from domain experts or produce only basic, limited ontologies. This gap hinders the full potential of relational data in semantic web applications and AI.
Methods
The core of RIGOR is an iterative, retrieval-augmented generation (RAG) pipeline that uses Large Language Models (LLMs) to convert relational database schemas into OWL ontologies. The process works table-by-table, guided by foreign key constraints:
- Retrieval: For each table, RIGOR retrieves relevant context from three sources:
- The expanding core ontology being built.
- The database schema itself, including table and column names, and any available natural language documentation.
- A repository of external domain ontologies (e.g., from BioPortal).
- Generation: A generative LLM (Gen-LLM) is prompted with the retrieved context to produce a "delta ontology" fragment for the current table. This fragment includes classes, object properties (for foreign keys), and data properties (for columns).
- Refinement: The generated delta ontology fragment is then validated and refined by a "Judge-LLM" (or a human expert). This step ensures semantic consistency, accuracy, and adherence to OWL 2 DL profiles.
- Integration: The refined delta ontology is merged into the core ontology, expanding its coverage. This iterative process continues until all tables in the database schema are processed.
This approach is described as a "hybrid recursive RAG" due to its multi-step retrieval and iterative nature.
Datasets
The authors evaluated RIGOR on two distinct real-world clinical databases:
- Real-World Database: A liver cancer registry containing 18 tables, 350 columns, and 15 foreign key relationships.
- ICU Database: A publicly available dataset from PhysioNet, comprising 32 tables, 559 columns, and 30 foreign keys.
For external knowledge, they utilized four biomedical ontologies from the BioPortal repository.
Key Insights and Ideas
- LLMs as Ontology Engineers: The paper demonstrates that LLMs, when guided by RAG and a structured iterative process, can effectively generate complex, semantically rich ontologies from structured database schemas, a task previously requiring significant human expertise.
- RAG is Crucial for Quality: Simply prompting LLMs with the schema is insufficient. The retrieval of context from the core ontology, external ontologies, and documentation is vital for producing accurate, complete, and semantically aligned ontologies.
- Iterative Refinement is Key: The "Judge-LLM" step is critical for ensuring the quality and validity of generated ontology fragments before they are integrated, preventing the propagation of errors and inconsistencies.
- Bridging Structured and Unstructured Data: RIGOR effectively bridges the gap between structured relational data and the rich semantic knowledge found in ontologies and natural language descriptions.
Key Results
- RIGOR consistently outperformed baseline (schema-only) and non-iterative (schema + external ontology in one pass) methods across all evaluated quality dimensions.
- The generated ontologies achieved high scores (average > 4.5/5) on ontology quality dimensions such as accuracy, completeness, conciseness, adaptability, clarity, and consistency, as evaluated by a Judge-LLM using competency questions (CQs).
- Structural analysis showed that DeepSeek 67B consistently produced the most structurally rich ontologies within the RIGOR framework.
- Semantic coverage analysis demonstrated strong alignment between ontology classes and database schema elements, with a class coverage rate of 72.4% for a sample ontology.
- The generated ontologies were syntactically valid and logically consistent, passing checks in Protégé and with the HermiT reasoner.
- The approach significantly reduced manual effort compared to traditional ontology engineering methods.
Limitations and Future Work
- LLM Hallucinations: While RAG helps mitigate hallucinations, the paper acknowledges that LLMs can still produce irrelevant or incorrect outputs, necessitating the Judge-LLM for validation.
- Prompt Engineering: The performance of the Gen-LLM and Judge-LLM is highly dependent on prompt design, which requires careful tuning.
- Scalability for Very Large Schemas: While the iterative approach is designed for scalability, extremely large and complex schemas might still pose computational challenges.
- Future Work: The authors suggest exploring more advanced RAG techniques, investigating different LLM architectures, and further automating the prompt engineering process. They also mention the potential for extending RIGOR to handle other data sources beyond relational databases.
Summary updated