LOGicalThought: Logic-Based Ontological Grounding of LLMs for High-Assurance Reasoning

Navapat Nananukul, Yue Zhang, Ryan Lee, Eric Boxer, Jonathan May, Vibhav Giridhar Gogate, Jay Pujara, Mayank KejriwalarXiv 2025 · DOI · arXiv

Introduction

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language processing, yet they often falter in "high-assurance" domains such as law, medicine, and tax compliance. These fields require rigorous, verifiable reasoning based on complex rule sets that frequently involve defeasible reasoning—rules that hold true unless specific exceptions apply. While standard prompting techniques like Chain-of-Thought (CoT) encourage step-by-step processing, they often lack formal grounding, leading to hallucinations or logical disconnects between explanations and final answers.

This paper introduces LOGicalThought (LOGT), a neurosymbolic framework designed to bridge the gap between natural language flexibility and formal logical rigor. By translating unstructured guidelines into symbolic graphs and executable logic programs, LOGT grounds LLM reasoning in a verifiable formal context.

Methodology: The LOGT Framework

The LOGT architecture transforms raw text into a dual-context representation to guide the LLM's final inference. The process consists of three primary stages:

  1. Symbolic Graph Context Generation: The system extracts an ontology (rule hierarchies), knowledge triples (scenario-specific facts), and natural language queries from the input guidelines and scenario.
  2. Logic-Based Context Construction: The symbolic graph is translated into an executable logic program using ErgoAI, a language optimized for defeasible reasoning. The framework includes a syntax correction loop to ensure the program compiles and executes, providing a set of formally proven facts.
  3. Hypothesis Evaluation: The LLM receives the original query alongside the Symbolic Graph and the results of the Logic Program execution. This grounded context enables the model to generate a final prediction (Entailment, Contradiction, or Neutral) supported by a structured, verifiable reasoning trace.
The LOGicalThought (LOGT) framework architecture, illustrating the transformation from natural language to symbolic graphs and executable logic programs.

Experimental Setup

The authors evaluated LOGT across four diverse Natural Language Inference (NLI) benchmarks:

  • ContractNLI: Legal agreement clauses.
  • SARA: Statutory reasoning based on the US Tax Code.
  • BioMedNLI: Clinical trial reporting.
  • Dungeons & Dragons NLI (DnD): A new, hand-crafted benchmark based on complex game rules to test dense logic and exceptions.

To ensure a robust evaluation, the benchmarks were augmented to specifically target three reasoning modes: Negation, Implication, and Defeasibility.

Results and Discussion

LOGT demonstrates significant performance improvements over standard baselines. On average, LOGT increased accuracy by +11.84% across all tested LLMs compared to the mean of all baselines, and by +4.41% over the strongest individual baseline.

Improved Reasoning Quality

A key finding is that LOGT fundamentally alters the model's reasoning behavior. As shown in Figure 2(b), LOGT traces contain a higher frequency of "apply_rule" and "fact_lookup" steps compared to CoT, indicating a shift from heuristic guessing to explicit rule application. Furthermore, LOGT significantly mitigates the "Right Reason, Wrong Answer" problem. In CoT, 41% of cases featured correct reasoning but an incorrect final label; LOGT reduced this disconnect to 28% (Figure 2(c)).

Performance analysis of LOGT: (a) Accuracy gains across benchmarks and reasoning modes; (b) Comparison of reasoning steps per trace; (c) Distribution of reasoning and prediction outcomes.

Model Scaling and Ablation

The framework acts as a powerful equalizer for smaller models. LLaMA-8B and Mistral-7B showed the most substantial gains when grounded in formal logic, allowing them to compete with much larger models on complex tasks. Ablation studies confirmed that while both symbolic graphs and executable logic programs contribute to performance, their combination yields the most robust results.

Conclusion and Limitations

LOGicalThought provides a scalable path toward trustworthy LLM reasoning in high-stakes environments. By forcing models to operate within a formal logical framework, it enhances both accuracy and verifiability. However, some domain-specific variances remain, such as slight performance dips in specific ContractNLI and SARA sub-tasks, suggesting that the translation layer may benefit from domain-specific refinement. Future work will focus on streamlining the multi-stage pipeline and improving the robustness of the logic program synthesis.

Summary updated

Abstract

High-assurance reasoning, particularly in critical domains such as law and medicine, requires conclusions that are accurate, verifiable, and explicitly grounded in evidence. This reasoning relies on premises codified from rules, statutes, and contracts, inherently involving defeasible or non-monotonic logic due to numerous exceptions, where the introduction of a single fact can invalidate general rules, posing significant challenges. While large language models (LLMs) excel at processing natural language, their capabilities in standard inference tasks do not translate to the rigorous reasoning required over high-assurance text guidelines. Core reasoning challenges within such texts often manifest specific logical structures involving negation, implication, and, most critically, defeasible rules and exceptions. In this paper, we propose a novel neurosymbolically-grounded architecture called LOGicalThought (LogT) that uses an advanced logical language and reasoner in conjunction with an LLM to construct a dual symbolic graph context and logic-based context. These two context representations transform the problem from inference over long-form guidelines into a compact grounded evaluation. Evaluated on four multi-domain benchmarks against four baselines, LogT improves overall performance by 11.84% across all LLMs. Performance improves significantly across all three modes of reasoning: by up to +10.2% on negation, +13.2% on implication, and +5.5% on defeasible reasoning compared to the strongest baseline.
BibTeX
@misc{nananukul2025logicalthought,
  author = {Navapat Nananukul and Yue Zhang and Ryan Lee and Eric Boxer and Jonathan May and Vibhav Giridhar Gogate and Jay Pujara and Mayank Kejriwal},
  title = {{LOGicalThought: Logic-Based Ontological Grounding of LLMs for High-Assurance Reasoning}},
  year = {2025},
  eprint = {2510.01530},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.01530},
}
Figure