Introduction
As Large Language Models (LLMs) evolve into increasingly complex "black boxes," their behaviors often transcend simple algorithmic explanations. To understand how these systems "think," reason, and interact, AI researchers have increasingly turned to the field of psychology. This shift treats LLMs not just as software, but as subjects for experimental inquiry. However, a recent large-scale bibliometric analysis reveals that while the bridge between AI and psychology is being built rapidly, it remains structurally incomplete.
The Problem: A Consensus of Misreading
The core challenge lies in the "Incomplete Bridge" problem. Interdisciplinary work requires a shared vocabulary, yet AI researchers often adopt psychological terms—such as "attention," "Theory of Mind," or "reasoning"—without fully embracing their original theoretical nuances. This leads to a "consensus of misreading," where flawed interpretations are codified into AI benchmarks. When these benchmarks become the standard for evaluation, they risk eroding scientific credibility and producing skewed conclusions about the true capabilities of artificial intelligence.
Mapping the Interdisciplinary Landscape
To quantify this phenomenon, researchers conducted a "science-of-science" analysis of over 25,000 LLM-related articles published between 2023 and 2025. By filtering for papers that specifically cited psychology literature, they identified a core corpus of 1,006 AI papers referencing 2,544 psychology publications.
Using the SPECTER transformer model and K-means clustering, the study mapped these works into distinct research clusters.
As shown in the figure above, LLM research topics were categorized into domains like Cognition & Reasoning, Human-Centric AI, and Model Adaptation. Psychology topics were similarly clustered into areas such as Social-Clinical, Psychometrics, and Neural Mechanisms.
The Citation Gap and the "GPT-4 Spark"
The data reveals a dramatic spike in citations to psychology research around March 2023, coinciding with the release of GPT-4. As models began to exhibit more "human-like" behaviors, the AI community felt an urgent need for psychological tools to measure them. However, this interest is highly selective.
The bipartite network in the figure above illustrates the uneven distribution of these connections. AI researchers heavily favor Psychometrics & Judgment/Decision-Making (JDM) and Neural Mechanisms. In contrast, domains like Education and Social-Clinical psychology remain largely underexplored, despite their potential relevance to AI safety and alignment.
Patterns of Misengagement
The study identifies four primary ways in which AI research misapplies psychological concepts:
- Conceptual Overgeneralization: Complex, multi-faceted theories are often reduced to "catch-all" labels. For instance, in "Theory of Mind" (ToM) research, AI papers frequently conflate first-order tasks (knowing what someone else thinks) with second-order tasks (knowing what someone thinks someone else thinks), despite the vastly different cognitive demands involved.
- Partial Citation: Researchers tend to cite a few "classic" papers while ignoring decades of modern refinement and nuance within psychology.
- The "Telephone" Effect: A significant pitfall is the reliance on secondary citations. AI researchers often cite other AI papers' interpretations of psychology rather than the original source, creating a closed loop where misconceptions are reinforced and amplified.
- Misinterpretation: There is a frequent conflation of biological mechanisms (e.g., neural activity) with complex social behaviors, leading to category errors in how model outputs are explained.
Conclusion: Toward Collaborative Parity
To fix the "shaky bridge," the authors advocate for Collaborative Parity. Rather than "retrofitting" psychological theories to explain results after an experiment is finished, AI researchers should engage with psychologists from the project's inception.
Building a robust interdisciplinary infrastructure—including shared datasets and measurement maps—is essential. Only through genuine collaboration can we move beyond superficial analogies and develop a rigorous science of LLM behavior that respects the depth of both computer science and psychology.
Summary updated