When human populations are surveyed on subjective topics, their responses naturally form a wide, high-variance distribution. Whether evaluating linguistic ambiguity, navigating ethical dilemmas, or expressing political opinions, human collective judgment is rarely unanimous. However, when large language models (LLMs) are deployed to simulate these populations or act as annotators, they exhibit a fundamental flaw: distributional collapse. Instead of mirroring the rich statistical variance of human beliefs, LLMs compress this diversity into low-entropy, highly confident, or homogenized outputs.
Understanding why this collapse happens—and how it manifests—is critical as models are increasingly used as proxies for human societies.
The Three Faces of Distributional Collapse
This collapse does not always look the same. Depending on the task framing and training history, models distort human distributions through three distinct behaviors:
1. Mode-Seeking and Caricaturization
In subjective tasks where humans are divided, models often assign nearly all probability mass to a single modal response. As demonstrated by Santurkar et al. in Whose Opinions Do Language Models Reflect?, models act as a caricature of a group's majority viewpoint rather than a representative sample of its pluralistic population. Similarly, Chiu et al. found that models converge on common cultural stereotypes when faced with multi-answer cultural questions in CulturalBench: A Robust, Diverse and Challenging Benchmark for Measuring LMs' Cultural Knowledge Through Human-AI Red-Teaming.
2. Regression to the Mean
When prompted to simulate populations across demographic or economic lines, models frequently exhibit a central tendency bias. In Are Large Language Models Chameleons?, Geng et al. show that LLMs gravitate toward stereotypical or average responses, failing to capture the long-tail diversity required for realistic social simulations. This generic, middle-of-the-road behavior is also highlighted by Bui et al. in Mixture-of-Personas Language Models for Population Simulation.
3. Binarization and Overconfidence
When faced with continuous scales or nuanced choices, models often push responses to extreme values. Wicke et al. observed this polarization on Likert-style scales in Exploring Spatial Schema Intuitions in Large Language and Vision Models. Even in complex ethical scenarios like trolley problems, Jin et al. note in Language Model Alignment in Multilingual Trolley Problems that models binarize choices by consistently selecting a single extreme option instead of reflecting the natural distribution of human moral preferences.
The Knowledge-to-Simulation Gap
Why do models fail to output variance even when they are exposed to diverse data during pre-training? A key insight comes from the distinction between knowing a distribution and simulating it.
In Benchmarking Distributional Alignment of Large Language Models, Guestrin et al. identify a knowledge-to-simulation gap. While an LLM can accurately state public opinion statistics in text (e.g., responding that "30% of people agree and 70% disagree"), it fails to replicate that distribution when prompted to act as individuals from that population or when its token probabilities are sampled.
Human Population Survey:
[ Agree (30%) ] [ Disagree (70%) ] --> High Variance
LLM Textual Knowledge:
"30% of people agree, 70% disagree" --> Correct Explicit Knowledge
LLM Simulated Sampling:
[ Disagree (100%) ] --> Collapsed Output
This occurs because standard training objectives and alignment pipelines are fundamentally at odds with uncertainty:
- Next-Token Optimization: Standard training minimizes cross-entropy loss. When training data contains multiple conflicting answers for the same context, the optimal strategy to minimize loss across the entire dataset often leads to allocating disproportionate probability to the dominant mode, creating peaky probability distributions. As Wilker Aziz demonstrates in Predict the Next Word: <Humans exhibit uncertainty in this task and language models _____>, LLMs fail to replicate the inherent aleatoric uncertainty of human language production.
- Alignment Compression: Reinforcement learning from human feedback (RLHF) tends to compress variance even further. By steering models toward a singular, low-entropy, consensus-seeking response, alignment pipelines inadvertently suppress legitimate human disagreement, a phenomenon observed by Lee et al. in Can Large Language Models Capture Dissenting Human Voices?.
Consequences Across Domains
The distributional variance gap creates severe bottlenecks across several core applications:
- Linguistic Uncertainty: When interpreting vague terms like "probably," human interpretations span a wide numerical range. However, Belém et al. show in Perceptions of Linguistic Uncertainty by Language Models and Humans that LLMs collapse onto specific, low-entropy values, erasing natural linguistic ambiguity.
- No-Consensus Tasks: In Arbiters of Ambivalence: Challenges of using LLMs in No-Consensus tasks, Bhaktipriya Radharapu highlights that when models are forced to act as judges or debaters in scenarios where humans deeply disagree, they abandon nuance and take decisive sides, artificially manufacturing a consensus.
- Educational and Cultural Modeling: In Can LLMs Solve Reading Comprehension Tests as Second Language Learners?, Horacio Saggion shows that while human learners distribute their incorrect guesses across multiple plausible distractors, LLMs concentrate high probability mass on a single incorrect option. This peaky overconfidence prevents models from accurately representing divided sentiment or multicultural perspectives, a limitation echoed by Zhao et al. in WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Models.
Bridging the Gap: From Prompts to Proxies
To overcome this collapse, researchers are moving away from naive sampling and toward structured architectural interventions.
One approach involves explicitly conditioning models on diverse personal profiles before generation. In Modeling Subjectivity in Cognitive Appraisal with Language Models, Zhou et al. demonstrate that while fine-tuned models excel at predicting the average human response, zero-shot prompting with explicit personality traits and demographics better replicates the natural spread of human disagreement.
Taking this further, Wang et al. introduce the Prompts to Proxies (P2P) framework in Prompts to Proxies: Emulating Human Preferences via a Compact LLM Ensemble. Instead of relying on a single model prompt to capture a population, P2P generates a functional basis of diverse personas and uses sparse regression to weight them. This ensemble approach counteracts the central tendency bias, allowing compact models to accurately emulate the true distributional fidelity of human viewpoints.
Key Foundational Work
Whose Opinions Do Language Models Reflect? (2023) — contributed 5 concept elements to this direction.
Are Large Language Models Chameleons? (2024) — contributed 3 concept elements to this direction.
Can Large Language Models Capture Dissenting Human Voices? (2023) — contributed 3 concept elements to this direction.