Abstract
Large Language Models (LLMs) often arrive at correct final answers through logically flawed intermediate steps, a phenomenon known as "hallucinating" reasoning. This paper introduces a new benchmark for evaluating the logical soundness of individual reasoning steps and proposes VANESSA (Verification of Answers by Natural deduction, Entailments, and Syntactic Sentence Analysis). VANESSA is a neuro-symbolic framework that combines neural language inference with a symbolic logic solver to provide transparent and verifiable proof traces. Our evaluation across multiple datasets demonstrates that VANESSA rivals black-box LLM verifiers while offering superior interpretability.
Introduction
Chain-of-Thought (CoT) prompting has significantly enhanced the reasoning capabilities of Large Language Models (LLMs) by encouraging step-by-step output. However, the internal logic of these steps often remains unverified; a model may produce a correct conclusion from incorrect premises or invalid logical leaps. Existing evaluation methods typically focus on the final answer or treat the entire reasoning chain as a single block.
This research argues for the necessity of verifying every individual step in a deductive chain based on two primary criteria:
- Groundedness: Ensuring every premise used in a step is present in the source text or previously derived conclusions.
- Validity: Ensuring the conclusion of a step logically follows from its stated premises.
The VANESSA Framework
To address the limitations of opaque neural verifiers and brittle symbolic systems, we introduce VANESSA. This neuro-symbolic approach bridges the gap between natural language flexibility and logical rigor through a three-stage process:
- Syntactic Parsing: Traditional NLP techniques decompose complex natural language sentences into atomic logical statements.
- Neural Entailment (NLI): An off-the-shelf NLI model (e.g., DeBERTa or LLaMa3) is employed strictly to determine entailment or contradiction between atomic statements, handling linguistic variations that stymie symbolic solvers.
- Natural Deduction Solver: These statements are processed by a custom symbolic solver. Notably, the solver utilizes modified natural deduction rules to avoid logical paradoxes like the "principle of explosion," ensuring the logic aligns with human linguistic intuition.
Reasoning Chain Verification Benchmark
We developed a comprehensive benchmark comprising 1,400 instances and over 5,000 manually annotated steps. This benchmark utilizes four foundational datasets:
- ProofWriter & EntailmentBank: Sound reasoning chains were programmatically corrupted via negation, removal, or hallucination of premises to test verifier robustness.
- ProntoQA & FOLIO: Reasoning chains generated by modern LLMs (Mixtral, LLaMa2, LLaMa3) were manually annotated for groundedness and validity.
Results and Discussion
Our experiments compare VANESSA against direct NLI baselines and LLM-to-First-Order-Logic (LINC) methods. Key findings include:
- Performance vs. Transparency: VANESSA (paired with LLaMa3) consistently ranks among the top-performing methods. It achieves F0.5 scores up to 0.98 on synthetic data and approximately 0.76 on complex real-world data (FOLIO).
- The Brittleness of Symbolic Logic: While LINC performs well on simple synthetic data, its performance degrades on complex text where LLM-based translation to logic becomes error-prone.
- Verifiable Proof Traces: Unlike black-box neural verifiers, VANESSA generates a symbolic proof tree for every approved step, providing a clear audit trail for the reasoning process.
Conclusion and Future Work
VANESSA demonstrates that a neuro-symbolic approach can provide the transparency required for high-stakes reasoning tasks without sacrificing significant performance. However, challenges remain, particularly regarding the "hallucination" of knowledge by the underlying NLI models. Future research will focus on strictly constraining these models to the provided context to further improve precision and reliability in automated reasoning verification.
Summary updated