Investigating the Robustness of Deductive Reasoning with Large Language Models

Fabian Hoppe, Filip Ilievski, Jan-Christoph KaloarXiv 2025 · DOI · arXiv

Introduction

Large Language Models (LLMs) have demonstrated remarkable capabilities in logical reasoning tasks. However, concerns persist regarding whether these models are performing genuine deduction or merely acting as "clever Hans" predictors—relying on statistical patterns and shortcuts. This study investigates the robustness of LLM reasoning when subjected to adversarial noise and counterfactual premises.

Methodology: Informal vs. Formal Reasoning

The researchers evaluated two distinct reasoning paradigms to determine how models handle logical tasks:

  1. Informal Reasoning: The model generates a direct answer or a natural language Chain-of-Thought (CoT).
  2. Autoformalisation: A hybrid approach where the LLM translates natural language into symbolic logic (e.g., First-Order Logic), which is then verified by a deterministic symbolic solver like Vampire.
Overview of the reasoning framework comparing informal reasoning and autoformalisation.

The framework tests these paradigms against three types of adversarial noise (Encyclopedic, Logical, and Tautological) and counterfactual shifts that negate original premises to contradict common sense, forcing the model to rely strictly on the provided text.

Key Results

The experimental results highlight a significant "Logic-Noise" trap. While autoformalisation provides a verifiable chain, it is more susceptible to logical-sounding distractions. Informal reasoning (CoT) proved more resilient at ignoring irrelevant information.

Performance comparison across different models, reasoning formats, and perturbation types.

The Counterfactual Barrier

A critical finding is the "Counterfactual Wall." Even advanced models like GPT-4o-mini experienced a 12-13% drop in accuracy when premises contradicted world knowledge. This suggests that LLMs struggle to prioritize provided logic over their internal "common sense" biases.

Impact of Noise and Syntax

  • Noise Sensitivity: Adding noise decreased autoformalisation accuracy by 4% to 11%, whereas informal reasoning remained relatively stable. Autoformalisation methods often try to incorporate irrelevant but "logical-sounding" sentences into the formal logic chain.
  • Syntax Choice: The choice of formal language (e.g., TPTP vs. FOL) significantly impacted performance. For instance, Llama 3.1 70B performed 12% better using TPTP syntax, likely due to its training data distribution.
  • Feedback Limitations: While providing syntax error messages helped models produce valid code, it did not consistently improve underlying reasoning accuracy. Models often fixed the syntax while leaving logical errors untouched.

Conclusion

The study concludes that current LLMs lack the robustness required for reliable deductive reasoning in noisy or counterfactual environments. While autoformalisation offers transparency and verifiability, its fragility to distraction and the persistent influence of internal model priors remain major hurdles for the development of truly logical AI systems.

Summary updated

BibTeX
@misc{hoppe2025investigating,
  author = {Fabian Hoppe and Filip Ilievski and Jan-Christoph Kalo},
  title = {{Investigating the Robustness of Deductive Reasoning with Large Language Models}},
  year = {2025},
  eprint = {2502.04352},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.04352},
}
Figure