Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities

Hans Hergen Lehmann, Jae Hee Lee, Steven Schockaert, Stefan Wermter

TL;DR

Large Language Models (LLMs) often possess the correct factual knowledge to answer a question but choose to ignore it in favor of superficial "shortcuts" or heuristics. This paper reveals that while larger models are better at strategically deciding when to trust their internal facts versus these surface cues, smaller models frequently act as "statistical parrots," relying on biases like entity popularity and mention order even when they "know" better.

What Problem Does This Paper Address?

There is a long-standing debate in AI: are LLMs "world models" that internalize structured knowledge, or are they "statistical parrots" that simply manipulate surface-level patterns?

The authors tackle this by looking at entity comparison. If you ask a model, "Which river is longer, the Danube or the Nile?", a true world model should retrieve the lengths and compare them. However, LLMs often get these questions wrong even when they can correctly state the individual lengths of both rivers if asked separately. This paper aims to untangle why this happens, identifying the specific heuristic biases that lead models astray and investigating how model scale and reasoning techniques (like Chain-of-Thought) change this behavior.

Methods

The researchers employed a highly controlled experimental setup to isolate reasoning from memorization:

  1. Pairwise Comparison vs. Numerical Extraction: They prompted models to compare two entities (e.g., "Which is more populous?") and separately asked the models to provide the actual numerical values for those entities.
  2. Balanced-Orthogonal Subset (BOS): To ensure that surface cues (like popularity) weren't simply correlated with the truth, they constructed a dataset where features were "orthogonal." For example, in half the cases, the more popular entity was actually the smaller one. This prevents the model from getting the "right" answer for the "wrong" reason.
  3. Meta-Predictor: They built a simple logistic regression model using only three surface cues (popularity, mention order, and semantic co-occurrence) to see if these "dumb" features could predict the LLM's choice better than the LLM's own numerical knowledge.
  4. Risk Ratios and Cohen’s d: These statistical measures were used to quantify how much each bias (like being mentioned first in a prompt) increased the likelihood of a model choosing that entity.

Datasets

The study used Wikidata to collect 10 different numerical attributes across diverse entity types:

  • Geography: River lengths, mountain elevations, city/country populations.
  • Science: Atomic numbers of elements.
  • Social/Culture: University enrollment, stadium capacity, Scoville heat units for peppers, and social media follower counts. The final dataset included up to 1,000 entities per attribute, sampled to create both "near-tie" challenges and clear-cut distinctions.

Key Insights and Ideas

  • "Knowing" is not "Using": Models frequently make pairwise predictions that contradict their own internal numerical knowledge. Numerical accuracy is consistently higher than comparison accuracy, meaning the model "knows" the facts but fails to apply them.
  • The Three Big Biases:
    • Mention Order: Models are strongly biased toward choosing the entity presented first or last in the prompt (position bias).
  • Entity Popularity: The "fame implies bigger" heuristic. Models often pick the more famous entity regardless of the facts.
  • Semantic Co-occurrence: Models are influenced by how often an entity appears near words like "large" or "tall" in their training data.
  • Scale as Strategy, Not Just Knowledge: The most surprising finding is that larger models (32B parameters) aren't necessarily "smarter" in terms of knowing more facts than 7B models. Instead, they are more meta-cognitive. They selectively rely on their internal knowledge when it is precise and fall back on heuristics only when their internal data is noisy or uncertain.
  • CoT as a Consistency Anchor: Chain-of-Thought (CoT) prompting forces models to verbalize their knowledge, which significantly narrows the gap between what the model "knows" and what it "chooses."

Key Results

  • Predictability: For smaller models (7-8B), the "meta-predictor" (using only surface cues) predicted model choices more accurately than the model’s own numerical estimates.
  • Internal Consistency: Larger models (32B) showed significantly higher internal consistency (agreement between their numbers and their choices) compared to smaller models.
  • CoT Impact: Prompting models to "think" before answering increased pairwise accuracy by 5–9 percentage points and improved internal consistency by a similar margin.
  • The 32B Advantage: While a 7B model might know the population of two cities more accurately than a 32B model, the 32B model is still more likely to win the comparison because it trusts its (less accurate) numbers more than the (distracting) surface cues.

Limitations and Future Work

  • Zero-Shot Focus: The study primarily looked at zero-shot performance. The authors note that fine-tuning on ranking tasks might change how these biases manifest.
  • Numerical Constraints: The analysis is limited to attributes with objective numerical ground truths. It remains to be seen how these heuristics affect subjective or non-numerical comparisons (e.g., "Which book is more influential?").
  • Mechanistic Interpretability: While the paper identifies what the models do, it doesn't fully explain the internal circuit-level mechanics of how a heuristic overrides a factual representation during the forward pass.
  • Future Direction: The authors suggest exploring whether updating the numerical knowledge "inside" a model (via editing) would actually fix its pairwise judgments, or if the heuristic shortcuts are too deeply embedded.

Summary updated

Abstract

Large Language Models (LLMs) are increasingly used for knowledge-based reasoning tasks, yet understanding when they rely on genuine knowledge versus superficial heuristics remains challenging. We investigate this question through entity comparison tasks by asking models to compare entities along numerical attributes (e.g., ``Which river is longer, the Danube or the Nile?''), which offer clear ground truth for systematic analysis. Despite having sufficient numerical knowledge to answer correctly, LLMs frequently make predictions that contradict this knowledge. We identify three heuristic biases that strongly influence model predictions: entity popularity, mention order, and semantic co-occurrence. For smaller models, a simple logistic regression using only these surface cues predicts model choices more accurately than the model's own numerical predictions, suggesting heuristics largely override principled reasoning. Crucially, we find that larger models (32B parameters) selectively rely on numerical knowledge when it is more reliable, while smaller models (7--8B parameters) show no such discrimination, which explains why larger models outperform smaller ones even when the smaller models possess more accurate knowledge. Chain-of-thought prompting steers all models towards using the numerical features across all model sizes.
BibTeX
@inproceedings{lehmann2026knowing,
  author = {Hans Hergen Lehmann and Jae Hee Lee and Steven Schockaert and Stefan Wermter},
  title = {{Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities}},
  booktitle = {EACL (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.222},
}

Loading related content...

Figure