Introduction
Large Language Models (LLMs) have demonstrated remarkable capabilities in logical reasoning tasks. However, concerns persist regarding whether these models are performing genuine deduction or merely acting as "clever Hans" predictors—relying on statistical patterns and shortcuts. This study investigates the robustness of LLM reasoning when subjected to adversarial noise and counterfactual premises.
Methodology: Informal vs. Formal Reasoning
The researchers evaluated two distinct reasoning paradigms to determine how models handle logical tasks:
- Informal Reasoning: The model generates a direct answer or a natural language Chain-of-Thought (CoT).
- Autoformalisation: A hybrid approach where the LLM translates natural language into symbolic logic (e.g., First-Order Logic), which is then verified by a deterministic symbolic solver like Vampire.
The framework tests these paradigms against three types of adversarial noise (Encyclopedic, Logical, and Tautological) and counterfactual shifts that negate original premises to contradict common sense, forcing the model to rely strictly on the provided text.
Key Results
The experimental results highlight a significant "Logic-Noise" trap. While autoformalisation provides a verifiable chain, it is more susceptible to logical-sounding distractions. Informal reasoning (CoT) proved more resilient at ignoring irrelevant information.
The Counterfactual Barrier
A critical finding is the "Counterfactual Wall." Even advanced models like GPT-4o-mini experienced a 12-13% drop in accuracy when premises contradicted world knowledge. This suggests that LLMs struggle to prioritize provided logic over their internal "common sense" biases.
Impact of Noise and Syntax
- Noise Sensitivity: Adding noise decreased autoformalisation accuracy by 4% to 11%, whereas informal reasoning remained relatively stable. Autoformalisation methods often try to incorporate irrelevant but "logical-sounding" sentences into the formal logic chain.
- Syntax Choice: The choice of formal language (e.g., TPTP vs. FOL) significantly impacted performance. For instance, Llama 3.1 70B performed 12% better using TPTP syntax, likely due to its training data distribution.
- Feedback Limitations: While providing syntax error messages helped models produce valid code, it did not consistently improve underlying reasoning accuracy. Models often fixed the syntax while leaving logical errors untouched.
Conclusion
The study concludes that current LLMs lack the robustness required for reliable deductive reasoning in noisy or counterfactual environments. While autoformalisation offers transparency and verifiability, its fragility to distraction and the persistent influence of internal model priors remain major hurdles for the development of truly logical AI systems.
Summary updated