A Comprehensive Evaluation of Cognitive Biases in LLMs

Simon Malberg, Roman Poletukhin, Carolin M. Schuster, Georg GrohNAACL 2025

TL;DR

This paper introduces a novel framework and a large-scale dataset for evaluating 30 cognitive biases in 20 state-of-the-art Large Language Models (LLMs). The study reveals that LLMs, like humans, exhibit a wide range of cognitive biases, underscoring the need for careful consideration when deploying them in high-stakes decision-making scenarios.

What Problem Does This Paper Address?

As LLMs become increasingly capable and integrated into complex reasoning and decision-making tasks, understanding their potential for flawed outputs is crucial. While biases like gender or ethical biases have been extensively studied, cognitive biases—systematic deviations from rational judgment—have only recently begun to be explored in LLMs. These biases can lead to significant risks, as LLMs might produce convincing but incorrect outputs. The authors aim to address the lack of a comprehensive, systematic, and scalable method for evaluating these cognitive biases across a broad range of LLMs and decision-making scenarios.

Methods

The authors developed a general-purpose framework for creating and conducting tests for LLMs. This framework involves:

  • Templates: Generalized descriptions of decision tasks with defined gaps for specific information.
  • Scenarios: Language sequences describing roles and environments to contextualize decision tasks.
  • Test Cases: Pairs of templates (control and treatment) designed to elicit specific biases by introducing subtle differences.
  • Functions: Processes to generate test cases by filling gaps in templates using LLM-generated insertions or custom value generators.
  • LLM Evaluation: A two-step prompt system (GEN for generation, DEC for decision-making) to interact with LLMs and collect their responses.
  • Bias Measurement: A universal metric to quantify the presence and strength of biases, adapted from existing cognitive science literature.

The bias selection process involved ranking cognitive biases by publication count in management contexts and then filtering for those testable with LLMs.

Datasets

The paper introduces a new benchmark dataset containing 30,000 cognitive bias tests. This dataset covers 30 cognitive biases and is generated across 200 unique managerial decision-making scenarios. The dataset was created using a GPT-40 LLM for scenario and gap generation. The authors also compare their dataset's diversity against two existing benchmarks: Echterhoff et al. (2024) and Tjuatja et al. (2024).

Key Insights and Ideas

  • LLMs are not immune to human cognitive biases: The study confirms and broadens previous findings, demonstrating that all 30 tested biases are present in at least some of the evaluated LLMs. This highlights a fundamental similarity between human and LLM decision-making processes.
  • A systematic framework is crucial for scalable bias evaluation: The authors' novel framework allows for reliable and large-scale generation of diverse tests, overcoming limitations of previous approaches that focused on fewer biases or single scenarios.
  • Bias is not uniformly distributed across LLMs: While all models exhibit biases, there's significant variation. Some models, like Gemini 1.5 Pro, show surprisingly low average biasedness, while others exhibit more pronounced biases.
  • Model characteristics don't perfectly predict bias: The study found no clear correlation between model size or general capability (as measured by Chatbot Arena scores) and the degree of biasedness, suggesting that architecture and training procedures play complex roles.
  • The dataset is diverse and largely non-toxic: The authors validated their dataset for correctness and diversity, finding it to be more diverse than existing benchmarks. Toxicity analysis indicated that the generated content is largely non-toxic.

Key Results

  • Evidence of all 30 tested biases: The evaluation found evidence for all 30 cognitive biases across the 20 LLMs.
  • Significant variation in biasedness: Models like Gemini 1.5 Pro were among the least biased, while others showed higher susceptibility.
  • Dataset Diversity: The new dataset demonstrated higher diversity compared to existing benchmarks, as measured by ROUGE, cosine similarities, Self-BLEU, and Remote-Clique distances.
  • High instruction-following accuracy: For verifiable instructions, the dataset achieved high accuracy (e.g., 100% for "Do not include any numbers" in Anchoring bias tests).
  • Low toxicity: The maximum toxicity scores for the dataset were generally low, with Stereotyping showing the highest score but still well below the maximum possible.

Limitations and Future Work

  • Limited scope of biases: While 30 biases were tested, over 150 cognitive biases are known in humans. Future work could expand the dataset to include more biases and domains.
  • LLM as data generator: The test cases were generated using a single LLM (GPT-40). While the framework is designed to mitigate this, future work could explore LLM-generated test cases for LLM evaluation more broadly.
  • Discrete decisions: The current evaluation focuses on discrete decisions. Future work could explore token probabilities for a more nuanced measurement of biases.
  • Framework generalizability: While the framework is theoretically generalizable, its versatility at scale across diverse domains needs further assessment.
  • LLM selection: The authors acknowledge that their evaluation of LLMs might be influenced by the specific models chosen and their availability at the time of research.

The authors encourage future research to build upon their framework and dataset to develop less biased and more reliable LLMs.

Summary updated

BibTeX
@inproceedings{malberg2025comprehensive,
  author = {Simon Malberg and Roman Poletukhin and Carolin M. Schuster and Georg Groh},
  title = {{A Comprehensive Evaluation of Cognitive Biases in LLMs}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure