TL;DR
This paper investigates the deductive reasoning capabilities of Large Language Models (LLMs) using syllogisms, a classic logic problem. They find that while LLMs exhibit human-like reasoning biases, supervised fine-tuning (SFT) significantly improves their performance and consistency, suggesting that form-based reasoning can be learned.
What Problem Does This Paper Address?
The rapid advancement of Large Language Models (LLMs) has raised a crucial question: have these models truly learned to reason, or are they merely sophisticated pattern matchers? Specifically, this paper focuses on syllogistic reasoning, a well-studied area of deductive logic that requires understanding the logical structure of arguments, independent of their content. Previous research has indicated that LLMs often exhibit "reasoning biases," such as favoring conclusions that align with real-world knowledge (content effect) and struggling with inferences where no conclusion logically follows. This work aims to systematically analyze how different training and prompting strategies (Chain-of-Thought, In-Context Learning, and Supervised Fine-Tuning) affect LLMs' ability to perform syllogistic reasoning, moving beyond simple accuracy metrics to understand the underlying reasoning processes.
Methods
The authors employ a multi-faceted approach to evaluate LLMs' syllogistic reasoning abilities:
- Zero-Shot Chain-of-Thought (ZS-CoT): Models are prompted to "think step-by-step" without any prior examples.
- In-Context Learning (ICL): Models are provided with a few examples (k=5) of syllogisms and their conclusions. They differentiate between
ICLin(examples from the same schema as the test case) andICLout(examples from different schemas). - Supervised Fine-Tuning (SFT): Models are trained on a dataset of syllogisms using standard cross-entropy loss.
Crucially, the analysis goes beyond accuracy by examining the consistency of generated conclusions (i.e., avoiding contradictory statements) and analyzing whether models adhere to logical form or rely on heuristics, drawing inspiration from cognitive science theories like the Atmosphere Heuristic.
Datasets
The study utilizes several datasets:
- Believable Syllogisms: Created by using real-world terms that form logically valid or invalid syllogisms where the conclusions are true.
- Unbelievable Syllogisms: Similar to believable syllogisms but constructed such that the conclusions violate real-world knowledge.
- Multi-Premise Syllogisms: Datasets with 3 and 4 premises, created by extending valid schemas with pseudo-words to test robustness to inference length.
- Training Data: Composed of pseudo-words to ensure models learn form-based reasoning rather than relying on semantic content.
Key Insights and Ideas
- Human-like Biases: Pre-trained LLMs exhibit reasoning patterns remarkably similar to humans, including a reluctance to state "nothing follows" for invalid inferences and a susceptibility to the "content effect" bias.
- SFT as a Game Changer: Supervised Fine-Tuning (SFT) is the most effective strategy for improving both accuracy and consistency, allowing models to overcome content biases and reliably handle valid and invalid inferences.
- ICL's Mixed Results: While In-Context Learning (ICL) boosts performance on valid inferences, it doesn't eliminate content effects or the difficulty with invalid syllogisms. It also tends to increase inconsistency.
- Atmosphere Heuristic as a Predictor: The Atmosphere Heuristic, a cognitive theory suggesting reasoners favor conclusions matching the "mood" of premises, effectively predicts the behavior of LLMs in ZS-CoT settings, explaining their tendency to avoid "nothing follows."
- Form vs. Content: The study strongly suggests that SFT, especially when using pseudo-words, is crucial for models to learn to reason based on logical form rather than relying on the semantic meaning of words.
Key Results
- ZS-CoT: LLMs like LLaMA-3 8B show significant accuracy on valid syllogisms (around 70%) but struggle with invalid ones (around 30%). They also exhibit a strong content effect bias.
- ICL:
ICLinandICLoutsettings improve accuracy on valid inferences but do not resolve the "nothing follows" issue and increase inconsistency. - SFT: Achieves near-perfect performance on both valid and invalid syllogisms for larger models, significantly reducing inconsistency and content effects. For instance, LLaMA-3 8B SFT reaches over 90% accuracy on believable syllogisms.
- Multi-Premise Reasoning: Performance generally decreases with more premises, but SFT remains the most robust strategy.
- Consistency: SFT dramatically reduces contradictory answers, especially for LLaMA, while ICL tends to increase them.
Limitations and Future Work
- Model Families: The analysis was limited to Pythia and LLaMA families. A broader investigation across more LLM architectures is needed.
- Model Sizes: Larger models (e.g., LLaMA-3 70B) were excluded due to computational constraints.
- Completeness vs. Consistency: While the paper focuses on consistency, a more thorough analysis of "completeness" (generating all correct conclusions) could be explored.
- Transferability of SFT: The authors question how well the reasoning abilities acquired through SFT on controlled datasets transfer to real-world, linguistically richer scenarios.
- Adversarial Studies: Further research could employ adversarial methods to rule out "shortcuts" learned by SFT models.
- Lexical Content Disruption: Experiments with disrupting lexical content using random real words, rather than just pseudo-words, could provide deeper insights.
Summary updated