Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning

Philipp Mondorf, Barbara Plank

Introduction

In the evaluation of Large Language Models (LLMs), accuracy is often treated as the ultimate benchmark. If a model consistently provides the correct answer to a logic puzzle, it is generally assumed to possess a robust internal reasoning mechanism. However, recent research from LMU Munich suggests that this assumption may be flawed. By applying frameworks from cognitive psychology, researchers have begun to look beyond the final output to examine the "thought traces" of LLMs. This study investigates whether these models employ the same inferential strategies as humans and whether their correct answers are the result of sound logic or mere statistical pattern matching.

Methodology

The study evaluated five open-access models: Llama-2 (7B, 13B, 70B), Mistral-7B-Instruct, and Zephyr-7B-β. To elicit reasoning processes, the researchers utilized zero-shot Chain-of-Thought (CoT) prompting, instructing the models to "think step by step."

The core of the analysis involved a manual review of 300 model responses by expert annotators. These responses were evaluated against five specific deductive strategies identified in a landmark 2002 study by Van der Henst et al., which characterized how humans solve propositional logic problems:

  1. Supposition Following: Making an assumption and tracing its logical consequences.
  2. Chain Construction: Building a linear sequence of "if-then" inferences.
  3. Compound Strategy: Combining multiple premises to derive a new, complex conclusion.
  4. Concatenation Strategy: Merging separate statements into a single, encompassing conclusion.
  5. Incremental Diagram: Constructing a comprehensive mental or literal map of all possibilities.

To ensure "content neutrality" and prevent models from relying on prior knowledge, the researchers used 12 propositional logic problems involving the presence or absence of colored marbles in a box. These problems utilized complex logical connectives such as biconditionals (if and only if) and exclusive disjunctions (either-or).

The Soundness Gap: Accuracy vs. Logic

The most significant finding of the study is the disconnect between accuracy and logical soundness. While models often reached the correct conclusion, their reasoning paths were frequently invalid. The researchers found only a moderate correlation ($r=0.45$) between getting the right answer and providing a sound logical argument.

A striking example of this was observed in Llama-2-7B, which achieved an accuracy rate of nearly 47% but demonstrated 0% sound reasoning. In these cases, the model essentially "guessed" correctly or arrived at the right answer through a series of logical fallacies. This highlights a "Clever Hans" risk in AI evaluation, where models mimic the appearance of logic without performing the underlying cognitive work.

Comparative Strategy Analysis

The study revealed that LLMs do indeed exhibit reasoning patterns similar to humans, but their preferences are heavily influenced by model scale and fine-tuning.

Model "Personalities"

Different model families showed distinct strategic preferences. For instance, Zephyr-7B acted as a "supposition specialist," utilizing that strategy in 60% of its reasoning traces. In contrast, Mistral-7B, despite being the base model for Zephyr, favored the Compound Strategy. This suggests that post-training (such as instruction tuning or RLHF) significantly alters the inferential path a model chooses to take.

The Symbolic Strategy

Interestingly, the researchers identified a strategy unique to LLMs that was not observed in the original human study: the Symbolic Strategy. Some models, particularly Llama-2-70B and Zephyr, occasionally translated natural language premises into formal logic symbols (e.g., $W \oplus R$) or truth tables. This indicates that LLMs may sometimes default to a more "computational" style of reasoning rather than the natural language heuristics typically used by humans.

Scaling and Soundness

Model size remains a primary driver of logical reliability. Llama-2-70B was the most sound model, producing valid reasoning in 31.7% of cases. This was a significant improvement over the 13B version (15%) and the 7B version (0%). Despite this improvement, even the most capable model failed to provide a sound argument two-thirds of the time.

Limitations and Challenges

The study identified several persistent bottlenecks in LLM reasoning:

  • Negation: Models consistently struggle with logical negations, often overlooking a "not" in a premise, which invalidates the entire reasoning chain.
  • Spatial Reasoning: LLMs rarely used the "Incremental Diagram" strategy, likely because it is a visual-spatial approach that is difficult to represent in a text-only format.
  • Scalability of Evaluation: The manual annotation required for this study is labor-intensive. Attempts to automate this process with a classifier showed poor generalization, highlighting the difficulty of evaluating reasoning at scale.

Conclusion

While LLMs are increasingly capable of solving complex logic puzzles, their internal processes remain fragile. They frequently mirror human-like strategies such as supposition and chaining, but they also develop unique symbolic approaches. However, the high frequency of "unsound" correct answers suggests that we must move beyond accuracy-based benchmarks. Future research, potentially involving Vision-Language Models (VLMs) to explore spatial reasoning strategies, will be essential in developing AI that doesn't just get the right answer, but gets it for the right reasons.

Summary updated

Abstract

Deductive reasoning plays a pivotal role in the formulation of sound and cohesive arguments. It allows individuals to draw conclusions that logically follow, given the truth value of the information provided. Recent progress in the domain of large language models (LLMs) has showcased their capability in executing deductive reasoning tasks. Nonetheless, a significant portion of research primarily assesses the accuracy of LLMs in solving such tasks, often overlooking a deeper analysis of their reasoning behavior. In this study, we draw upon principles from cognitive psychology to examine inferential strategies employed by LLMs, through a detailed evaluation of their responses to propositional logic problems. Our findings indicate that LLMs display reasoning patterns akin to those observed in humans, including strategies like supposition following or chain construction. Moreover, our research demonstrates that the architecture and scale of the model significantly affect its preferred method of reasoning, with more advanced models tending to adopt strategies more frequently than less sophisticated ones. Importantly, we assert that a model’s accuracy, that is the correctness of its final conclusion, does not necessarily reflect the validity of its reasoning process. This distinction underscores the necessity for more nuanced evaluation procedures in the field.
BibTeX
@inproceedings{mondorf2024comparing,
  author = {Philipp Mondorf and Barbara Plank},
  title = {{Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.508},
}
Figure