Abstract
Large Language Models (LLMs) are increasingly used for knowledge-based reasoning tasks, yet understanding when they rely on genuine knowledge versus superficial heuristics remains challenging. We investigate this question through entity comparison tasks by asking models to compare entities along numerical attributes (e.g., "Which river is longer, the Danube or the Nile?"), which offer clear ground truth for systematic analysis. Despite having sufficient numerical knowledge to answer correctly, LLMs frequently make predictions that contradict this knowledge. We identify three heuristic biases that strongly influence model predictions: entity popularity, mention order, and semantic co-occurrence. For smaller models, a simple logistic regression using only these surface cues predicts model choices more accurately than the model's own numerical predictions, suggesting heuristics largely override principled reasoning. Crucially, we find that larger models (32B parameters) selectively rely on numerical knowledge when it is more reliable, while smaller models (7–8B parameters) show no such discrimination, which explains why larger models outperform smaller ones even when the smaller models possess more accurate knowledge. Chain-of-thought prompting steers all models towards using the numerical features across all model sizes.
Introduction
A long-standing debate in AI asks whether LLMs are "world models" that internalize structured knowledge or "statistical parrots" that simply manipulate surface-level patterns. This paper addresses this by examining entity comparison. If an LLM is asked which of two rivers is longer, a true world model should retrieve and compare their lengths. However, LLMs often fail these comparisons even when they can correctly state the individual lengths of both entities if asked separately. This study identifies the specific heuristic biases that lead models astray and investigates how model scale and reasoning techniques like Chain-of-Thought (CoT) influence this behavior.
Methodology
The researchers employed a controlled experimental setup to isolate reasoning from memorization:
- Pairwise Comparison vs. Numerical Extraction: Models were prompted to compare two entities and separately asked to provide their actual numerical values.
- Balanced-Orthogonal Subset (BOS): To ensure surface cues like popularity were not merely correlated with the truth, the researchers constructed a dataset where features were orthogonal. For example, in half the cases, the more popular entity was actually the smaller one.
- Meta-Predictor: A logistic regression model was built using only three surface cues (popularity, mention order, and semantic co-occurrence) to see if these features could predict LLM choices better than the LLM's own numerical knowledge.
- Statistical Measures: Risk ratios and Cohen’s d quantified how much each bias increased the likelihood of a model choosing a specific entity.
The study utilized Wikidata to collect 10 numerical attributes across geography, science, and social culture, including river lengths, atomic numbers, and social media follower counts.
The Gap Between Knowledge and Application
A primary insight is that "knowing" a fact does not equate to "using" it. Models frequently make pairwise predictions that contradict their internal numerical knowledge. As shown in Figure 1, numerical accuracy is consistently higher than comparison accuracy, indicating that models often possess the facts but fail to apply them during reasoning.
Heuristic Biases in Model Reasoning
Three primary heuristic biases were identified as major influences on model predictions:
- Mention Order: Models exhibit a strong position bias, often choosing the entity presented first or last in the prompt.
- Entity Popularity: A "fame implies magnitude" heuristic where models favor the more famous entity regardless of the facts.
- Semantic Co-occurrence: Models are influenced by how frequently an entity appears near descriptors like "large" or "tall" in training data.
Figure 2 quantifies these biases, showing that for many models, surface cues like mention order and popularity have a massive impact on the final choice.
The Strategic Advantage of Scale
The study reveals that larger models (32B parameters) are not necessarily "smarter" in terms of raw factual knowledge than 7B models. Instead, they are more meta-cognitive: they selectively rely on internal knowledge when it is precise and fall back on heuristics only when their internal data is noisy. In contrast, smaller models often act as statistical parrots, where surface cues predict their choices more accurately than their own numerical estimates.
Conclusion and Future Work
Chain-of-Thought prompting acts as a consistency anchor, significantly narrowing the gap between what a model knows and what it chooses. However, the study is limited to objective numerical truths and zero-shot performance. Future research should explore whether updating internal numerical knowledge via model editing fixes pairwise judgments or if heuristic shortcuts are too deeply embedded in the model's architecture.
Summary updated