HeriKGQA: Grounded Knowledge-Graph Question Answering over Cultural Heritage Sensor Data

Lingxiao Kong, Vasileios Arampatzakis, Vasileios Sevetlidis, George Pavlidis, Zeyd BoukhersarXiv 2026 · arXiv

TL;DR

HeriKGQA is a novel question-answering system designed to monitor cultural heritage sites by combining the natural language strengths of Large Language Models (LLMs) with the mathematical precision of Knowledge Graphs. By restricting the LLM to parsing questions and phrasing answers—while offloading all calculations to a deterministic "grounding layer"—it achieves a remarkable 99% accuracy, significantly outperforming standard RAG-based approaches.

What Problem Does This Paper Address?

Cultural heritage sites like the House of Masks in Delos are increasingly monitored by vast sensor networks tracking temperature, humidity, and vibration. While this data is vital for conservation, querying it usually requires database expertise.

LLMs seem like the perfect natural-language interface, but they fail "silently" when faced with raw sensor records. As shown in Figure 1, a standard LLM might interleave readings from two different sensors, misinterpreting a steady state as a rapid temperature jump and raising a false alarm. Furthermore, LLMs are notoriously unreliable at performing complex aggregations (like averages over thousands of rows) or spatial reasoning (like calculating distances between sensor nodes). For a conservator deciding if a monument is at risk, a fluent but mathematically incorrect answer is worse than no answer at all.

Figure 1: Example challenge of QA over heritage data. Over raw text, the LLM mixes two sensors’ read- ings and raises a false alarm (left); HeriKGQA evaluates sensors separately over the knowledge graph (right).

Methods

The authors introduce HeriKGQA, which operates on a core principle: the LLM neither computes nor writes the code that computes.

The system architecture, detailed in Figure 2, consists of several key components:

  1. Sensor Knowledge Graph (KG): Raw data is mapped to a heritage-specific ontology (PANOPTES). This ensures every reading is explicitly linked to its specific sensor, site, and timestamp, preventing the "mixing" errors seen in raw text.
  2. Semantic Parsing: An LLM maps the user's question into a constrained JSON "intent." This intent specifies filters, aggregations, and target classes without writing free-form code.
  3. Deterministic Compilation: This intent is automatically compiled into a SPARQL query. Because the compilation is deterministic, it avoids the syntax errors common when LLMs try to write SPARQL directly.
  4. Grounding Layer: This is the "math engine." It handles spatial distances (using Haversine formulas), temporal facts, and complex anomaly rule evaluations that are difficult to express in pure SPARQL or risky for an LLM to calculate.
  5. Error Correction: The system uses "intent voting" (running multiple parses and picking the most frequent) and "answer verification" (checking every number in the final answer against the raw data) to ensure total reliability.
Figure 2: Overview of HeriKGQA. Bottom: sensor readings are mapped through a heritage ontology, extended with anomaly rules, into a virtual knowledge graph G. Middle: the LLM (purple) only parses the question and verbalises the answer, while SPARQL and the grounding layer (teal) compute every number from G. Top: voting over intents, retrying failed queries, and verifying answer numbers guard the LLM steps.

Datasets

The researchers evaluated their system using:

  • HeriKGQA Benchmark: A new dataset of 500 questions (basic, spatial, temporal, anomaly-related, and follow-up) based on real data from five heritage sites.
  • MultiTQ: A general temporal KGQA benchmark used to test the system's portability to other domains.
  • Real-World Deployment: 80 questions logged from actual site managers and partners using the ARGUS platform chatbot.

Key Insights and Ideas

  • Computation Offloading is Key: The most significant finding is that "grounding"—moving math out of the LLM—is non-negotiable for sensor data. Without the grounding layer, accuracy on anomaly detection questions plummeted from 99% to 66%.
  • Metadata Preservation: By using a Knowledge Graph instead of flat text, the system maintains the "provenance" of every data point. This allows for per-sensor rule evaluation even when sensors share the same channel names (e.g., multiple "temp_1" sensors).
  • Smaller Backbones Suffice: Interestingly, because the LLM is only used for parsing and verbalization, the system's accuracy remained at 99% even when swapping a massive 120B parameter model for a much smaller 27B parameter one (Qwen-27B).
  • Transparency Builds Trust: The system includes an expandable "References" section in its chatbot interface (Figure 3), allowing experts to inspect the exact SPARQL query and retrieved data used to generate an answer.
Figure 3: HeriKGQA deployed in the chatbot of the ARGUS platform. Users select a pilot site (top left; here the House of Masks, Delos), shown as a 3D model with its sensor nodes, and ask questions in the chat panel (right). Each answer states the time of the latest reading used and its data source, with an expandable References section holding the parsed intent, SPARQL query, and retrieved data behind it, and a feedback prompt.

Key Results

HeriKGQA demonstrated superior performance across all metrics compared to three strong baselines (TextRAG, TableRAG, and Interactive-KBQA):

  • Accuracy: Reached 99.0% overall, which is 18.2 points higher than the strongest baseline.
  • Speed: It ran nearly 4 times faster than the Interactive-KBQA baseline (12.0s vs 46.7s median latency).
  • Reliability: It achieved 100% accuracy on spatial questions, where baselines often failed due to imprecise coordinate arithmetic.
Table 1: Quantitative comparison showing HeriKGQA's dominance in accuracy and competitive latency.

Limitations and Future Work

Despite its success, the authors note several limitations:

  • Template Dependency: The primary benchmark uses template-generated questions. While evaluation on real user questions was promising (78.8% accuracy), real-world phrasing is more variable and complex.
  • Graph-Specific Logic: The grounding layer currently requires custom code for specific types of spatial or anomaly analysis. Making this layer more "plug-and-play" for new ontologies is a goal for future work.
  • Large Result Sets: To avoid errors from sampling, the system currently declines to answer questions that would require processing more than 1,000 rows of data in the final verbalization step.

In conclusion, HeriKGQA provides a robust blueprint for building trustworthy AI interfaces for critical infrastructure and heritage monitoring, proving that for high-stakes data, the best role for an LLM is as a translator, not a calculator.

Summary updated

Abstract

Cultural heritage sites are increasingly monitored by sensor networks, but querying their data requires database expertise. Large Language Models (LLMs) offer a natural-language interface, yet over raw sensor records they lose track of which sensor, site, and time each value belongs to, and silently miscompute. We present HeriKGQA, a knowledge-graph question-answering method that confines the LLM to interpreting questions and phrasing answers, leaving computation to SPARQL and deterministic code. Questions are parsed into intents constrained by a heritage ontology aligning W3C SOSA/SSN with CIDOC-CRM, compiled into SPARQL, and answered from query results and temporal, spatial, and anomaly facts derived by a grounding layer, with error correction guarding each LLM step. On 500 questions over real data from five heritage sites, HeriKGQA reaches 99.0% accuracy, 18.2 points above the strongest of three baselines using the same LLM and receiving the same anomaly rules and answer conventions, with comparable answer quality while running almost four times faster. Without grounding, accuracy drops by 12.2 points and on anomaly questions from 99% to 66%. We report lessons from development and live deployment.
BibTeX
@misc{kong2026herikgqa,
  author = {Lingxiao Kong and Vasileios Arampatzakis and Vasileios Sevetlidis and George Pavlidis and Zeyd Boukhers},
  title = {{HeriKGQA: Grounded Knowledge-Graph Question Answering over Cultural Heritage Sensor Data}},
  year = {2026},
  eprint = {2611.15904},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15904},
}
Figure