Introduction
Do Large Language Models (LLMs) "think" like humans? While previous research has focused on comparing the final outputs of AI to human behavior, this study investigates the internal "trajectories" of a model's forward pass. The researchers hypothesize that the way information evolves across Transformer layers mirrors human cognitive patterns, particularly the tendency to favor intuitive-but-incorrect answers before settling on a correct one.
The Process Gap: Layers as a Proxy for Time
Traditionally, LLMs have been treated as "black boxes." If a model assigns a high probability to a sentence that humans find easy to read, it is deemed "human-like." However, this approach ignores the process of thought. Humans process information over time, often experiencing "competitor interference" where a tempting but wrong answer slows down decision-making. Transformers, conversely, process information across layers. This paper asks whether the "layer-time" of a Transformer forward pass maps onto the "real-time" of human cognition.
Methodology
The authors analyzed 20 open-source models (including Llama-3.1, Gemma-2, and Falcon-3) using a technique called the Logit Lens. This allows researchers to "peek" at intermediate representations at every layer. They introduced several novel metrics to quantify these dynamics:
- Change of Mind (CoM): Measures the degree to which a model initially favors an "intuitive" (incorrect) answer in early layers before switching to the correct one.
- Time to Decision (TTD): The specific layer where the model begins to consistently prefer the correct answer.
- Dynamic AUC Measures: Calculations of the "Area Under the Curve" for entropy (uncertainty) and confidence across the model's entire depth.
These metrics were used in Bayesian mixed-effects regressions to predict human behavioral data, such as reaction times and mouse-tracking paths, across domains including capital city recall, animal categorization, and logical syllogisms.
Key Results and Insights
The "Two-Stage" Signature
Just like humans, models often exhibit a "dual-process" signature. In early layers, they are frequently lured by "intuitive" competitors (e.g., choosing Chicago as the capital of Illinois because it is the most populous city). Later layers then "correct" this initial impulse. The study found that these internal dynamics (CoM and TTD) are significantly better predictors of human processing load (reaction time) than the final probability assigned to an answer.
Predictive Power and the "Middle-Child" Effect
Dynamic measures significantly improved the prediction of human processing indicators compared to static baselines. Interestingly, the largest models were not always the most human-like. Mid-sized models often showed the strongest signs of "two-stage" processing, while the largest models sometimes became less like humans in specific tasks, such as animal categorization.
Furthermore, the researchers identified a functional split: a model's internal uncertainty (entropy) effectively predicts human processing load, while its internal confidence is a better predictor of human accuracy.
Conclusion and Future Work
While layers are not a perfect proxy for continuous human time, the alignment found in this study suggests that Transformer dynamics can serve as explicit tools for simulating human mental processes. Future work may involve designing architectures specifically intended to mimic human processing trajectories or developing "early exit" strategies for AI that stop processing once a confidence threshold is reached, mimicking human snap judgments. This research moves us closer to understanding AI not just by what it says, but by how it "thinks."
Summary updated