Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach

Zheyuan Zhang, Jifan Yu, Juanzi Li, Lei Hou

Introduction: Beyond the Leaderboard

Traditional evaluations of Large Language Models (LLMs) often rely on aggregate scores from benchmarks like MMLU. While these leaderboards indicate that a model is performing well, they fail to explain how it processes information or where its cognitive boundaries lie. This "black-box" approach masks a significant paradox: LLMs can solve complex professional problems while simultaneously failing at basic logic or factual recall.

Researchers from Tsinghua University have addressed this gap by applying "Educational Diagnostic Assessment"—a method rooted in pedagogical psychology—to map the internal knowledge structures of LLMs. By diagnosing specific strengths and deficits, this research moves toward a more nuanced understanding of machine intelligence.

Methodology: Educational Diagnostic Assessment

The study adopts a two-pronged diagnostic approach: Deficit Assessment, which identifies knowledge gaps across domains, and Error Assessment, which analyzes patterns in both final answers and the reasoning chains provided by the models.

The Educational Diagnostic Assessment framework applied to LLMs.

To standardize this evaluation, the researchers utilized Bloom’s Taxonomy, categorizing tasks into four Knowledge Types (Factual, Conceptual, Procedural, and Meta-knowledge) and five Cognitive Dimensions (Remember, Understand, Apply, Analyze, and Evaluate). Three OpenAI models—text-davinci-003, ChatGPT, and GPT-4—were tested in zero-shot settings, occasionally supplemented with retrieved "study materials" to measure the impact of contextual information.

The MoocRadar Dataset

The core of the evaluation is MoocRadar, a dataset of 8,453 human test questions across STEM, Social Science, and Humanities. Unlike general benchmarks, every question in MoocRadar is meticulously annotated with its Bloom’s Taxonomy category. To provide a comparative baseline, the researchers integrated real student performance data from the MOOCCubeX platform, allowing for a direct "Likeness" comparison between AI and human cognitive structures.

Cognitive Mapping: The "Intermediate Dip" and STEM Paradox

The diagnostic results revealed several distinct cognitive patterns in LLMs that differ from human expectations:

  1. The STEM Paradox: LLMs consistently outperformed humans in STEM subjects but struggled more with Humanities and Social Sciences. Despite being natural language models, they were more prone to hallucinations regarding "long-tail" concepts in the humanities, while excelling at the structured, logical frameworks of STEM.
  2. The "Intermediate Dip": Both models and humans showed a performance drop in the "intermediate" cognitive dimensions of Apply and Analyze. These tasks require multi-step deduction and the connection of multiple concepts, making them more error-prone than simple recall (Remember) or high-level judgment (Evaluate).
  3. The Primacy Effect: A significant cognitive bias was observed where models were disproportionately likely to select the first option in a multiple-choice list. While ChatGPT showed some robustness, GPT-3.5 and GPT-4 performance plummeted when correct answers were moved to the end of the list.
Heatmaps comparing the cognitive knowledge structures of OpenAI models against human students.

Performance and Human-Likeness

The study quantified how closely LLM cognitive structures mirror human ones using a "Likeness" score. As models evolved from GPT-3.5 to GPT-4, their knowledge distribution became increasingly similar to the human baseline (GPT-4 correlation: 0.474 vs. GPT-3.5: 0.262).

In terms of raw performance, GPT-4 led the models with a score of 0.657, which improved to 0.687 when relevant context was provided. However, even the most advanced models still trailed the human average of 0.746. A critical finding was the lack of internal consistency: models frequently generated correct explanations but chose the wrong option, or vice versa, highlighting a disconnect in their internal reasoning chains.

Conclusion and Future Directions

While LLMs are rapidly approaching human-like cognitive structures, significant gaps remain in multi-step reasoning and consistency. The authors note that their study was limited to closed-source OpenAI models and that dataset imbalances in certain Bloom categories (like Meta-knowledge) require further investigation.

Future research will focus on using these diagnostic insights to optimize training protocols. Rather than just maximizing accuracy, the goal is to shape LLMs into more balanced and robust cognitive agents, capable of reliable reasoning across all dimensions of human knowledge.

Summary updated

Abstract

Large Language Models (LLMs) have not only exhibited exceptional performance across various tasks, but also demonstrated sparks of intelligence. Recent studies have focused on assessing their capabilities on human exams and revealed their impressive competence in different domains. However, cognitive research on the overall knowledge structure of LLMs is still lacking. In this paper, based on educational diagnostic assessment method, we conduct an evaluation using MoocRadar, a meticulously annotated human test dataset based on Bloom Taxonomy. We aim to reveal the knowledge structures of LLMs and gain insights of their cognitive capabilities. This research emphasizes the significance of investigating LLMs’ knowledge and understanding the disparate cognitive patterns of LLMs. By shedding light on models’ knowledge, researchers can advance development and utilization of LLMs in a more informed and effective manner.
BibTeX
@inproceedings{zhang2023exploring,
  author = {Zheyuan Zhang and Jifan Yu and Juanzi Li and Lei Hou},
  title = {{Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.findings-emnlp.111},
}
Figure