Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset

Kentaro Ozeki, Risako Ando, Takanobu Morishita, Hirohiko Abe, Koji Mineshima, Mitsuhiro OkadaACL 2024 · DOI

TL;DR

This paper introduces NeuBAROCO, a new bilingual (English/Japanese) dataset for evaluating syllogistic reasoning in Large Language Models (LLMs). They found that LLMs exhibit reasoning biases similar to humans, particularly in cases where conclusions are neither entailed nor contradicted by premises. The study also highlights that LLMs' primary limitations lie in their reasoning process rather than just interpreting syllogisms.

What Problem Does This Paper Address?

While Large Language Models (LLMs) have shown impressive capabilities, their ability to perform accurate deductive reasoning, a cornerstone of classical AI, is still an active area of research. Humans are known to exhibit various "reasoning biases" – systematic errors in logic that deviate from normative reasoning. These biases, studied extensively in cognitive science, are crucial to understand because they can reveal underlying cognitive mechanisms. However, evaluating LLMs for these human-like reasoning biases, especially in natural language, has not been systematically addressed. This paper aims to bridge this gap by investigating whether LLMs exhibit similar reasoning biases to humans when tackling syllogistic reasoning, a fundamental form of deductive inference.

Methods

The authors introduce a new task called Translate-and-Explain which leverages Chain-of-Thought (CoT) prompting. This method encourages LLMs to first translate syllogisms into formal logical expressions (predicate logic or set theory) and then explain their reasoning process. This allows for a finer-grained analysis to pinpoint whether errors stem from misinterpreting the syllogism's language or from flaws in the reasoning process itself. They also employ standard Multiple-Choice and Natural Language Inference (NLI) tasks for evaluation.

Datasets

The core contribution is the NeuBAROCO dataset, a new bilingual (English/Japanese) syllogistic reasoning dataset. It's derived from the BAROCO test, originally designed for psychological experiments on human reasoning. NeuBAROCO includes:

  • NLI Task: 790 problems designed to classify inferences into entailment, contradiction, or neutral.
  • Multiple-Choice Task: 95 problems, mirroring those used in human psychological experiments. The dataset is annotated with various reasoning biases, such as belief bias, conversion errors, and atmosphere effects, allowing for direct comparison with human performance data.

Key Insights and Ideas

  • LLMs Mimic Human Biases: The study's most striking finding is that LLMs, particularly advanced ones like GPT-4, exhibit reasoning biases remarkably similar to those observed in human participants. This suggests that the underlying mechanisms driving these errors might be shared.
  • Reasoning Process is the Bottleneck: The Translate-and-Explain task revealed that LLMs' primary limitations are not in understanding the natural language of syllogisms but in the actual reasoning process. Even when translations were highly accurate, reasoning errors persisted.
  • Neutral Cases are Tough: LLMs struggle most with syllogisms where the conclusion is neither entailed nor contradicted by the premises. This "neutral" category appears to be the most challenging, even for top-performing models.
  • Bias Types Matter: Specific biases, like conversion errors and atmosphere effects (related to the form of sentences), significantly reduce LLM accuracy, mirroring human performance patterns on these types of problems.

Key Results

  • Multiple-Choice: GPT-4 achieved high accuracy (up to 95% in Japanese), sometimes surpassing human performance (53% in Japanese). However, performance varied across different bias types, with incongruent cases being harder in English.
  • NLI Task: GPT-4 showed strong performance (around 70-79% overall), with few-shot prompting improving accuracy. Models struggled most with neutral inferences (around 30 points lower than entailment/contradiction). Accuracies were significantly lower for problems prone to conversion and atmosphere biases.
  • Translate-and-Explain: Translation accuracy into predicate logic reached nearly 100% for GPT-4. However, reasoning accuracy remained lower, indicating the reasoning process itself is the main challenge.

Limitations and Future Work

  • Closed-Source Models: The study relied on closed-source LLMs (GPT-3.5, GPT-4), meaning precise details about their parameters and training data are unknown, limiting full reproducibility.
  • Dataset Overlap: There's a possibility that some syllogistic reasoning patterns, especially symbolic ones, might have been present in the LLMs' training data.
  • Human-LLM Comparison Methodology: The authors note that a standardized methodology for comparing LLM accuracy to human accuracy is still developing.
  • Future Work: The paper suggests expanding evaluations to more complex natural language inferences beyond syllogisms, such as conditional and spatial reasoning, as the identified biases may generalize. Further research is needed to understand the reasoning capabilities of LLMs more deeply.

Summary updated

BibTeX
@inproceedings{ozeki2024exploring,
  author = {Kentaro Ozeki and Risako Ando and Takanobu Morishita and Hirohiko Abe and Koji Mineshima and Mitsuhiro Okada},
  title = {{Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.950},
}
Figure