KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge

Alex Robertson, Huizhi Liang, Mahbub Gani, Rohit Kumar, Srijith Rajamohan

TL;DR

KGHaluBench is a new dynamic benchmark that moves beyond static, multiple-choice evaluations to probe the "breadth and depth" of LLM knowledge using Wikidata. By distinguishing between an LLM's ability to recognize a concept (breadth) versus its ability to recall specific facts about it (depth), the framework provides a more nuanced map of where and why models hallucinate.

What Problem Does This Paper Address?

Evaluating factual accuracy in Large Language Models (LLMs) is a moving target. Existing benchmarks often suffer from three major flaws:

  1. Static Nature: Fixed datasets are quickly incorporated into training data (data contamination), making them tests of memory rather than reasoning or "live" knowledge.
  2. Surface-Level Probing: Many benchmarks use multiple-choice or short-answer formats that don't reflect the complex, long-form queries users actually pose.
  3. Popularity Bias: Models are naturally better at discussing "Head" entities (like Taylor Swift) than "Tail" entities (like a niche 19th-century painter). Standard benchmarks often fail to account for this difficulty gap, leading to skewed performance metrics.

The authors introduce KGHaluBench to solve these issues by dynamically generating multifaceted questions from a Knowledge Graph (KG) and statistically scaling scores based on question difficulty.

Methods

The KGHaluBench framework consists of two main modules:

1. Dynamic Question Generation: The system selects a "focal entity" from Wikidata and retrieves three specific facts (triples) about it. It then uses a template to generate an open-ended prompt requiring the LLM to provide a 200-word overview including those three facts.

  • Difficulty Scaling ($Q_d$): To address popularity bias, the authors use Item Response Theory (IRT). They calculate a difficulty score based on entity prominence (page views, site links) and relation complexity (how rare the specific facts are). This allows them to calculate Weighted Accuracy ($W_a$), which rewards models more for answering difficult, niche questions correctly.

2. Automated Verification Pipeline: Evaluating long-form text is notoriously hard. KGHaluBench uses a two-layer approach:

  • Entity-Level Filter: Uses semantic similarity (via Jina-Embeddings) to check if the model's response actually describes the correct entity. If it doesn't, it’s flagged as a "Breadth of Knowledge" hallucination ($Halu_{BOK}$).
  • Fact-Level Check: A tri-stage pipeline (NLI Entailment $\rightarrow$ LLM Entailment $\rightarrow$ Expert LLM Decision) verifies if the three specific facts requested are present and correct. Failures here are flagged as "Depth of Knowledge" hallucinations ($Halu_{DOK}$).

Datasets

  • Wikidata: The primary source, providing a massive pool of ~118 million entities and their relational facts.
  • Wikipedia: Used to fetch entity descriptions for the semantic similarity check in the verification pipeline.
  • KGHaluBench: The paper introduces this as a reproducible benchmarking framework rather than a single static dataset.

Key Insights and Ideas

  • Breadth vs. Depth: The most significant insight is the decomposition of hallucinations. A model might know who a person is (Breadth) but fail on when they were born or what they studied (Depth).
  • The "Knowledge Frontier" Gap: As models scale, their "Breadth" hallucinations ($Halu_{BOK}$) drop dramatically—they recognize almost everything. However, their "Depth" hallucinations ($Halu_{DOK}$) persist much longer. This suggests that simply making models larger helps them recognize more topics, but doesn't necessarily make them more precise about the details of those topics.
  • Constructive Abstention: The authors found that some models (like GPT-5-Mini) have high "abstention" rates. While this reduces hallucinations, over-abstaining can make a model less useful. The paper argues for "constructive abstention," where a model admits what it doesn't know while guiding the user to reliable sources.

Key Results

The authors evaluated 25 frontier models, including GPT-4/5, Claude, Gemini, and Llama 3.

  • Top Performer: GPT-5 (2025-08-07) achieved the highest Weighted Accuracy ($W_a$) at 65.60%.
  • Proprietary vs. Open-Source: Proprietary models generally outperformed open-source models (average $W_a$ of 55.94% vs 48.32%), though the gap is narrowing. GLM-4.5 was a standout open-source performer, beating several proprietary models.
  • Hallucination Trends: In smaller models (8B-32B), $Halu_{BOK}$ was as high as 54.75%. In the largest proprietary models, this dropped to ~11.91%. However, $Halu_{DOK}$ (fact-level errors) only dropped by about 24% across that same scale, highlighting the difficulty of achieving "depth."

Limitations and Future Work

  • Source Bias: Wikidata is unevenly populated across different cultures and languages, which may introduce bias into the benchmark.
  • Verification Errors: While the automated pipeline achieved high agreement with humans (up to 87.74%), it is not perfect and can still misinterpret nuanced long-form responses.
  • Future Directions: The authors suggest expanding the benchmark to include domain-specific Knowledge Graphs (e.g., medical or legal) and improving the reasoning capabilities of the automated verifiers.

Reference: Robertson, A., et al. (2026). KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge. arXiv:2602.19643v1.

Summary updated

Abstract

Large Language Models (LLMs) possess a remarkable capacity to generate persuasive and intelligible language. However, coherence does not equate to truthfulness, as the responses often contain subtle hallucinations. Existing benchmarks are limited by static and narrow questions, leading to limited coverage and misleading evaluations. We present KGHaluBench, a Knowledge Graph-based hallucination benchmark that assesses LLMs across the breadth and depth of their knowledge, providing a fairer and more comprehensive insight into LLM truthfulness. Our framework utilises the KG to dynamically construct challenging, multifaceted questions, whose difficulty is then statistically estimated to address popularity bias. Our automated verification pipeline detects abstentions and verifies the LLM's response at both conceptual and correctness levels to identify different types of hallucinations. We evaluate 25 frontier models, using novel accuracy and hallucination metrics. The results provide a more interpretable insight into the knowledge factors that cause hallucinations across different model sizes. KGHaluBench is publicly available to support future developments in hallucination mitigation.
BibTeX
@misc{robertson2026kghalubench,
  author = {Alex Robertson and Huizhi Liang and Mahbub Gani and Rohit Kumar and Srijith Rajamohan},
  title = {{KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge}},
  year = {2026},
  eprint = {2602.19643},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.19643},
}
Figure