Cognitive Foundations for Reasoning and Their Manifestation in LLMs

Priyanka Kargupta, Shuyue Stella Li, Haocheng Wang, Jinu Lee, Shan Chen, Orevaoghene Ahia, Dean Light, Thomas L. Griffiths, Max Kleiman-Weiner, Jiawei Han, Asli Celikyilmaz, Yulia Tsvetkov

TL;DR

Large language models (LLMs) frequently excel at complex tasks yet stumble on simpler variants, revealing a fundamental divergence from human-like reasoning. This paper introduces a cognitive science-backed taxonomy of 28 elements to map the internal reasoning traces of LLMs, demonstrating that models rely on rigid, sequential processing where humans use flexible meta-cognition. By automatically scaffolding these successful human-like cognitive structures at test-time, the authors achieve performance boosts of up to 66.7% on complex, ill-structured problems.

What Problem Does This Paper Address?

Despite their impressive benchmarks, LLMs suffer from counterintuitive generalization failures—solving a grueling math problem but failing on a trivial modification. Because current evaluation paradigms only score the final output, they cannot distinguish between genuine reasoning and sophisticated memorization.

To resolve this "measurement crisis," the authors bridge the gap between cognitive science and AI. They address a core question: What specific cognitive building blocks are models actually deploying when they "think," and how do they differ from human thought?

Figure 1: An example of cognitive elements present in a reasoning trace for building a LEGO spaceship. We characterize the different elements along the four dimensions of our taxonomy, as shown in Table 1.

Methods

The authors formalize a unified taxonomy of 28 cognitive elements categorized into four dimensions:

  1. Reasoning Invariants: Core validity criteria (e.g., logical coherence, compositionality).
  2. Meta-Cognitive Controls: Executive functions that monitor and adjust strategies (e.g., self-awareness, strategy selection).
  3. Reasoning Representations: How knowledge is organized (e.g., hierarchical, causal, or spatial structures).
  4. Reasoning Operations: Actions that transform representations (e.g., verification, abstraction, backtracking).

Using this taxonomy, they build an automated framework using frontier models (validated by humans) to perform fine-grained, span-level annotations on thousands of reasoning chains. They map these chains into heterogeneous transition graphs, using Normalized Pointwise Mutual Information (NPMI) to calculate which sequences of cognitive elements directly correlate with a correct answer.

Finally, they introduce test-time reasoning guidance. This technique takes the successful consensus subgraphs for a given problem type and automatically injects structural prompts to steer the model's generation strategy.

Datasets

The empirical analysis spans 192,709 reasoning traces across 18 open-weight models and three modalities:

  • Text: 169,792 traces sampled from GeneralThought (verifiable tasks) and ClaimSpect (nuanced real-world claims).
  • Audio: 4,917 traces from BLAB, MMAR, and MMAU-Pro.
  • Image: 18,000 multimodal traces from Zebra-CoT.
  • Human Baseline: 54 high-quality "think-aloud" verbal reasoning traces from 18 educated human participants, providing a direct point of comparison.

Key Insights and Ideas

  • The "Inverse Deployment" Trap: On well-structured problems (like algorithms), LLMs deploy a rich, diverse set of cognitive strategies. However, on complex, "ill-structured" problems (like open-ended dilemmas or multi-step diagnoses), models paradoxically narrow their strategy, defaulting to rigid sequential processing and surface-level enumeration. Humans do the exact opposite.
  • The Execution Gap: Models frequently attempt meta-cognitive behaviors like verification and logical coherence (appearing in up to 91% of traces), but their presence weakly correlates with success. LLMs note contradictions but fail to effectively fix them, leading to repetitive loops.
  • Research Community Blindspots: A meta-analysis of 1,598 AI reasoning papers reveals that the ML community heavily focuses on easily quantifiable elements like sequential organization (55%) and decomposition (60%), while severely neglecting success-critical elements like self-awareness (16%).
Figure 2: Distribution of cognitive element presence across 1,598 arXiv LLM Reasoning papers. Partially present indi- cates that there is evidence that the element was considered in the design (motivation, method, evaluation) of the paper but was not the primary focus. Present indicates that there is evidence that the element was a conscious and significant design decision. Details provided in Section 5.

Key Results

  • Massive Gains via Structural Steering: Applying test-time reasoning guidance to capable models yielded massive accuracy improvements on complex tasks: +66.7% for Qwen3-7B on diagnostic problems and +60.0% on dilemmas for both Qwen3-14B and R1-Distill-Qwen-32B.
  • The Capability Threshold: Smaller or less sophisticated models (<8B parameters) showed severe performance degradation (up to -72%) when steered. This suggests a strict threshold where explicit scaffolding overloads a model's limited cognitive capacity if it lacks robust instruction-following capabilities.

Limitations and Future Work

The authors note that while test-time guidance successfully alters model behavior, it remains unclear whether it elicits genuine latent reasoning capabilities or simply helps the model more effectively retrieve cached, pre-trained shortcuts. Additionally, the framework was only tested on transformer-based architectures.

A notable failure mode occurred during "Design" problems, where steering caused catastrophic failure (0% accuracy) across all models, highlighting that some highly ambiguous problem spaces require entirely new scaffolding paradigms. Future work will focus on using these cognitive metrics to shape RL reward models directly during training.

Summary updated

Abstract

Large language models (LLMs) solve complex problems yet fail on simpler variants, suggesting they achieve correct outputs through mechanisms fundamentally different from human reasoning. To understand this gap, we synthesize cognitive science research into a taxonomy of 28 cognitive elements spanning reasoning invariants, meta-cognitive controls, representations for organizing reasoning & knowledge, and transformation operations. We introduce a fine-grained evaluation framework and conduct the first large-scale empirical analysis of 192K traces from 18 models across text, vision, and audio, complemented by 54 human think-aloud traces, which we make publicly available. We find that models under-utilize cognitive elements correlated with success, narrowing to rigid sequential processing on ill-structured problems where diverse representations and meta-cognitive monitoring are critical. Human traces show more abstraction and conceptual processing, while models default to surface-level enumeration. Meta-analysis of 1.6K LLM reasoning papers reveals the research community concentrates on easily quantifiable elements (sequential organization: 55%, decomposition: 60%) but neglecting meta-cognitive controls (self-awareness: 16%) that correlate with success. Models possess behavioral repertoires associated with success but fail to deploy them spontaneously. Leveraging these patterns, we develop test-time reasoning guidance that automatically scaffold successful structures, improving performance by up to 66.7% on complex problems. By establishing a shared vocabulary between cognitive science and LLM research, our framework enables systematic diagnosis of reasoning failures and principled development of models that reason through robust cognitive mechanisms rather than spurious shortcuts, while providing tools to test theories of human cognition at scale.
BibTeX
@misc{kargupta2025cognitive,
  author = {Priyanka Kargupta and Shuyue Stella Li and Haocheng Wang and Jinu Lee and Shan Chen and Orevaoghene Ahia and Dean Light and Thomas L. Griffiths and Max Kleiman-Weiner and Jiawei Han and Asli Celikyilmaz and Yulia Tsvetkov},
  title = {{Cognitive Foundations for Reasoning and Their Manifestation in LLMs}},
  year = {2025},
  eprint = {2511.16660},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.16660},
}
Figure