Abstract
The reasoning abilities of Large Language Models (LLMs) are becoming a central focus of study in NLP. In this paper, we consider the case of syllogistic reasoning, an area of deductive reasoning studied extensively in logic and cognitive psychology. Previous research has shown that pre-trained LLMs exhibit reasoning biases, such as content effects, avoid answering that no conclusion follows, align with human difficulties, and struggle with multi-step reasoning. We contribute to this research line by systematically investigating the effects of chain-of-thought reasoning, in-context learning (ICL), and supervised fine-tuning (SFT) on syllogistic reasoning, considering syllogisms with conclusions that support or violate world knowledge and with multiple premises. Crucially, we go beyond the standard focus on accuracy, with an in-depth analysis of the conclusions generated by the models. Our results suggest that the behavior of pre-trained LLMs can be explained by heuristics studied in cognitive science and that both ICL and SFT improve model performance on valid inferences, although only the latter can mitigate most reasoning biases while being consistent.
Introduction: The "Soft Reasoning" Debate
The core debate in modern AI is whether Large Language Models (LLMs) have latently learned robust reasoning skills or are merely sophisticated "stochastic parrots" relying on superficial patterns. A classic test for this is syllogistic reasoning (e.g., "All mammals are animals; all dogs are mammals; therefore, all dogs are animals").
Humans are notoriously bad at this when logic conflicts with reality—a phenomenon called the content effect. For example, we find it harder to reject a logically invalid conclusion if it is "believable" (e.g., "Some pets are cats"). This study investigates whether LLMs exhibit similar struggles, specifically focusing on three weaknesses: recognizing invalid inferences, resisting content effects, and maintaining performance across longer inference chains.
Methodology and Datasets
The researchers compared three primary learning strategies across two model families (Pythia and LLaMA-2/3):
- Zero-Shot Chain-of-Thought (ZS-CoT): Prompting the model to "think step-by-step" without prior examples.
- In-Context Learning (ICL): Providing five examples of syllogisms in the prompt. They tested both "in-schema" (matching the test logic) and "out-of-schema" examples.
- Supervised Fine-Tuning (SFT): Training models on a dataset of syllogisms using pseudo-words (e.g., "All shucts are gloogy") to force the model to ignore real-world knowledge and focus purely on logical form.
The evaluation utilized 64 distinct premise pairs (27 valid, 37 invalid), sets of believable vs. unbelievable conclusions, and newly constructed 3- and 4-premise syllogisms. To avoid fact-memorization, the SFT training set used 4,000 nonsense words.
Key Insights: Heuristics and Inconsistencies
One of the most striking findings is that LLM behavior in zero-shot settings is highly correlated with the Atmosphere Heuristic. This cognitive science theory suggests that reasoners pick conclusions based on the "mood" (quantifiers like 'all' or 'some') of the premises rather than logic. For instance, if both premises use "some," the model often assumes the conclusion must also use "some."
Furthermore, while In-Context Learning (ICL) improves accuracy on valid syllogisms, it introduces a significant problem: inconsistency. When provided with examples, models become more likely to generate contradictory conclusions within the same response—for example, stating a specific conclusion and then immediately following it with "Nothing follows."
Results and Conclusions
- Performance vs. Humans: LLaMA-3 70B achieved 58.89% accuracy on valid syllogisms, surpassing the human average of ~44%. However, it still lagged behind humans on invalid inferences (32.5% vs. ~41%), showing a strong reluctance to state that no valid conclusion exists.
- The Power of SFT: Fine-tuning on abstract pseudo-words was the most effective method. For LLaMA-3 8B, SFT boosted accuracy to 96.25% and virtually eliminated the content effect bias, reducing the "believability gap" from 25% to nearly zero.
- Model Scale: Smaller models (e.g., Pythia-160M) were almost entirely unable to perform the task, while larger models showed stronger correlations with human-like reasoning patterns.
In summary, while pre-trained LLMs act as "soft reasoners" prone to human-like heuristics, Supervised Fine-Tuning on abstract data can "detach" models from content bias, instilling a more robust understanding of logical form. Future work should investigate if this logical purity generalizes to complex, natural language reasoning beyond controlled templates.
Summary updated