Detecting LLM Fact-conflicting Hallucinations Enhanced by Temporal-logic-based Reasoning

Ningke Li, Yahui Song, Kailong Wang, Yuekang Li, Ling Shi, Yi Liu, Haoyu WangarXiv 2025 · DOI · arXiv

Introduction

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, they remain prone to Fact-conflicting Hallucinations (FCH)—instances where the model generates content that directly contradicts established real-world facts. Detecting these errors is a persistent challenge because current benchmarks often rely on static datasets that quickly become obsolete or focus on simple fact retrieval that models can bypass through memorization without true reasoning.

To address these gaps, researchers have introduced DROWZEE, an automated metamorphic testing framework. DROWZEE is designed to expose deep-seated reasoning flaws by generating complex, time-sensitive test cases using Metric Temporal Logic (MTL) and logic programming.

The DROWZEE Framework

DROWZEE moves away from manual labeling toward an end-to-end automated pipeline for test generation and evaluation. The framework follows a structured approach to transform raw data into rigorous logical tests.

The DROWZEE pipeline, illustrating the flow from data extraction to response evaluation.
  1. Knowledge Extraction: The system crawls Wikipedia and Wikidata to build a base of factual triples (Subject-Predicate-Object), ensuring the knowledge base can be updated to reflect current reality.
  2. Logic-based Expansion: Using Prolog, DROWZEE applies reasoning rules—such as transitive, inverse, and composite relations—to transform simple facts into complex logical chains.
  3. Temporal Logic (MTL): A core innovation is the integration of Metric Temporal Logic. DROWZEE generates questions involving operators like "finally," "globally," and "until" within specific time windows (e.g., "Did event X happen within 3 years of event Y?").
  4. Metamorphic Testing & Semantic Oracles: Instead of simple string matching, DROWZEE evaluates the model's reasoning steps. It uses "semantic-aware oracles" to compare the graph structure of the LLM’s reasoning against the ground truth using Jaccard similarity.

Knowledge vs. Inference Hallucinations

A key contribution of this research is the distinction between two types of hallucinations: Erroneous Knowledge and Erroneous Inference.

Examples of (a) Erroneous Knowledge Hallucination and (b) Erroneous Inference Hallucination.
  • Erroneous Knowledge: The model possesses incorrect underlying facts (e.g., claiming Haruki Murakami won the Nobel Prize in 2016).
  • Erroneous Inference: The model may have access to the correct facts but fails the logical step required to connect them (e.g., knowing both authors' awards but incorrectly concluding they won the same one).

Interestingly, the study found that inference errors are a major culprit, accounting for roughly half of the detected hallucinations in advanced models.

Key Results and Insights

The researchers evaluated nine state-of-the-art models, including GPT-4, Llama 3, and Mistral. Several critical insights emerged:

  • Hallucination Prevalence: In non-temporal tasks, hallucination rates ranged from 24.7% (GPT-4) to nearly 60% (Mistral-7B). GPT-4 remains the most reliable, while smaller open-source models struggle significantly with complex logic.
  • Temporal Sensitivity: LLMs are notably worse at discerning sequential logic and temporal intervals. While a model might know specific dates, it often fails to calculate if those dates fall within a relative time window.
  • The "Honesty" Problem: When faced with knowledge occurring after their training cutoff, LLMs rarely admit ignorance. Instead, they confidently fabricate facts to satisfy the logic of the prompt.
  • Domain Variance: Hallucination rates are higher in specialized fields like Mathematics and Natural Sciences, likely due to the intricate interrelationships between entities in those domains.

Conclusion

DROWZEE provides a robust, automated method for identifying the logical and temporal failures of LLMs. By focusing on metamorphic relations and semantic oracles, the framework achieves a 99.6% precision in identifying FCH cases. Future work aims to explore "white-box" methods to understand the internal states leading to these breakdowns and to develop training techniques that encourage model honesty when faced with unknown facts.

Summary updated

Abstract

Large language models (LLMs) face the challenge of hallucinations -- outputs that seem coherent but are actually incorrect. A particularly damaging type is fact-conflicting hallucination (FCH), where generated content contradicts established facts. Addressing FCH presents three main challenges: 1) Automatically constructing and maintaining large-scale benchmark datasets is difficult and resource-intensive; 2) Generating complex and efficient test cases that the LLM has not been trained on -- especially those involving intricate temporal features -- is challenging, yet crucial for eliciting hallucinations; and 3) Validating the reasoning behind LLM outputs is inherently difficult, particularly with complex logical relationships, as it requires transparency in the model's decision-making process. This paper presents Drowzee, an innovative end-to-end metamorphic testing framework that utilizes temporal logic to identify fact-conflicting hallucinations (FCH) in large language models (LLMs). Drowzee builds a comprehensive factual knowledge base by crawling sources like Wikipedia and uses automated temporal-logic reasoning to convert this knowledge into a large, extensible set of test cases with ground truth answers. LLMs are tested using these cases through template-based prompts, which require them to generate both answers and reasoning steps. To validate the reasoning, we propose two semantic-aware oracles that compare the semantic structure of LLM outputs to the ground truths. Across nine LLMs in nine different knowledge domains, experimental results show that Drowzee effectively identifies rates of non-temporal-related hallucinations ranging from 24.7% to 59.8%, and rates of temporal-related hallucinations ranging from 16.7% to 39.2%.
BibTeX
@misc{li2025detecting,
  author = {Ningke Li and Yahui Song and Kailong Wang and Yuekang Li and Ling Shi and Yi Liu and Haoyu Wang},
  title = {{Detecting LLM Fact-conflicting Hallucinations Enhanced by Temporal-logic-based Reasoning}},
  year = {2025},
  eprint = {2502.13416},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.13416},
}
Figure