TL;DR
When large language models (LLMs) encounter epistemic uncertainty (a lack of knowledge), they do not fail randomly—they rely on a predictable hierarchy of fallback behaviors: repetitions, degenerate text, and hallucinations. As models become stronger (via parameter scaling, extended pretraining, or instruction tuning) or when random decoding is introduced, they trade easily detectable repetition loops for harder-to-detect hallucinations.
What Problem Does This Paper Address?
Language models notoriously exhibit three frustrating failure modes:
- Looping verbatim repetitions,
- Producing degenerate text patterns, and
- Generating convincing hallucinations.
Previously, researchers studied these phenomena in isolation. This paper asks: Are these disparate failure modes connected, and what governs how an LLM chooses between them?
The authors frame all three as fallback behaviors triggered under epistemic uncertainty (when the model simply does not know the answer). By analyzing these behaviors under controlled conditions, the paper uncovers a unified behavioral spectrum and demonstrates how scaling and decoding parameters push models along this spectrum.
Methods
Rather than relying on noisy post-hoc confidence estimates, the authors induce epistemic uncertainty by construction:
- Controlled Generation Demands: Prompting models to generate 25 items for factual lists where fewer than 15 valid answers exist, or prompting them about fictitious entities.
- Sequential Fact Tagging: Generations are parsed into atomic claims and categorized sequentially. The first appearance of a claim is marked as correct or hallucination; duplicate appearances are labeled as repetitions.
- Statistical Ordering Analysis: They introduce $\text{ShiftScore}$ to evaluate whether fact categories follow an ordered transition (Repetition $\to$ Hallucination $\to$ Correct) and validate this ordering using Mann-Whitney U-tests ($p < 10^{-8}$).
- Continuous Degeneration Tracking: They measure lexical collapse via a token-level $\text{DiversityScore}$ (unique tokens divided by sequence length) across generation length.
Evaluations span multiple open-source model families (Pythia [70M–12B], OLMo [1B–7B], Llama 2 [7B, 70B], and Llama 3 [8B, 70B]) across base, intermediate pretraining, and instruction-tuned checkpoints.
Datasets
The study evaluates structured list generation and open-ended text across four datasets:
- TriviaFacts (TRIV): A newly curated, exhaustive benchmark of 95 list questions with non-ambiguous, short answers (e.g., colors in Olympic rings). Ground-truth sets are capped at $\le 10$ items to guarantee uncertainty when models are forced to supply 25 answers.
- QAMPARI (QAMP): A 100-question sample from Amouyal et al. (2023) representing open-domain list queries with answers sourced from Wikipedia.
- FakeQAMPARI (FQAMP): A new synthetic dataset where subject entities from QAMPARI are replaced with plausible-sounding fictional names to test model behavior under 100% false premises.
- BioGeneration (BIO): 125 Wikipedia entities stratified across five popularity levels (from very rare to very frequent) to test open-ended biography generation evaluated via FActScore.
Key Insights and Ideas
- The Fallback Hierarchy: Fallback behaviors sit on a complexity spectrum: $$\text{Verbatim Repetitions (Simplest)} \longrightarrow \text{Degenerate Text} \longrightarrow \text{Hallucinations (Most Complex)}$$
- Model Strength Replaces Loops with Hallucinations: Increasing model capacity (more parameters, longer pretraining, or instruction tuning) allows models to avoid naive text looping, but channels that uncertainty into coherent confabulations instead.
- Phase Shifts within a Single Sequence: During long-form generation, even the best models exhibit this progression in reverse over time: they produce correct facts first, slide into hallucinations as parametric memory runs dry, and finally deteriorate into repetitive or degenerate text.
- Random Sampling Trades One Symptom for a Worse One: Nucleus or temperature sampling successfully breaks repetitive loops, but does so by sharply escalating the rate of plausible hallucinations.
- Prompting Models to Abstain Fails: When explicitly instructed to "stop when you cannot recall additional facts," instruction-tuned models actually generate more hallucinations, exposing severe miscalibration—they hallucinate because they are overconfidently "sure."
Key Results
- Model Scaling on TriviaFacts: In the Pythia family, moving from 70M to 12B parameters drops average repetitive facts from 16.3 to 12.2 while driving hallucinations up from 3.0 to 8.0.
- Extreme Uncertainty (FakeQAMPARI): When asked about non-existent entities, models virtually never abstain. Models $>1\text{B}$ parameters hallucinated on $>30\%$ of slots—more than double the $\sim 15\%$ rate in sub-billion models.
- Temperature Trade-offs: For Pythia-12B on TriviaFacts, sweeping temperature $\tau$ from $0.0$ to $1.0$ lowered repetitions from 12.2 down to 2.0, but surged hallucinations from 8.0 to 17.2 while diminishing correct facts from 4.1 to 2.7.
- Temporal Deterioration: Wikipedia human baselines maintain a steady $\text{DiversityScore}$ between $0.7$ and $0.8$. In contrast, generated text degrades continuously, dropping below the 0.5 diversity threshold within just $\sim 150$ tokens.
Limitations and Future Work
- Scope of Uncertainty: The study focuses exclusively on epistemic uncertainty (lack of factual recall). It does not test aleatoric uncertainty (inherent ambiguities or stochastic prompts).
- Domain Focus: Evaluations center on natural-language knowledge extraction; fallback mechanics during code synthesis or complex mathematical reasoning remain unstudied.
- Decoding Exploration: While temperature and nucleus ($p$) sampling were analyzed, advanced repetition penalties, contrastive decoding, and downstream agentic guardrails were left for future exploration.
Summary updated