When we ask a group of humans a subjective question—such as "is a fake dollar still a dollar?" or "how much do you trust the healthcare system?"—we expect a distribution of answers. Some people will be certain, others hesitant, and many will disagree entirely. This pluralism is the bedrock of social science.
However, as researchers increasingly use Large Language Models (LLMs) as "silicon subjects" to replace human participants, they have encountered a fundamental discrepancy. While an LLM can mimic a human-like response, it struggles to mimic a human-like population. Instead of reflecting the broad spectrum of human belief, models tend to exhibit "variance collapse," converging on a single, overconfident, or stereotypical mode.
The Certainty Bias and Mode Seeking
The core of the problem lies in how LLMs handle ambiguity. In human populations, uncertainty is expressed through a wide distribution of opinions. In contrast, Kathryn Davidson and Hayley Ross have shown that LLMs exhibit a "certainty bias," producing binary or extreme probability outputs for questions that elicit a wide range of nuanced responses from humans Is artificial intelligence still intelligence? LLMs generalize to novel adjective-noun pairs, but don't mimic the full human distribution.
This behavior is often described as "mode-seeking." When faced with a question that has multiple valid cultural or moral answers, models often collapse onto the most common or stereotypical response. Yu Ying Chiu and colleagues found that frontier models see a nearly 29% drop in accuracy when moving from single-answer cultural questions to those with multiple correct answers, as they fail to recognize alternative valid practices CulturalBench: A Robust, Diverse and Challenging Benchmark for Measuring LMs' Cultural Knowledge Through Human-AI Red-Teaming.
Why Alignment Flattens Diversity
If the base training of a model captures the "whole sea" of internet data, why do the resulting models feel so narrow? Evidence suggests that the very process of making models "safe" and "helpful" through Reinforcement Learning from Human Feedback (RLHF) exacerbates this collapse.
Sonia Krishna Murthy and Jennifer Hu argue that alignment "flattens" internal representations, making aligned models significantly less conceptually diverse than their unaligned counterparts One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversity. By rewarding the model for producing the single "best" or "safest" response, we inadvertently train it to ignore the dissenting voices and edge cases that exist in the training data. This results in what Shibani Santurkar and Percy Liang describe as a "caricature" of a population—a model that reflects a narrow, often "liberal, wealthy, and highly educated" demographic rather than a pluralistic society Whose Opinions Do Language Models Reflect?.
The Knowledge-to-Simulation Gap
A striking finding in this research area is that models often "know" about diversity even when they cannot "be" diverse. Carlos Guestrin and Tatsunori Hashimoto identified a "knowledge-to-simulation gap": a model can accurately describe a group's opinion distribution in text (e.g., generating a JSON object representing a survey result) far better than it can actually simulate individuals from that group through repeated sampling Benchmarking Distributional Alignment of Large Language Models.
This suggests that the information about human variance is present in the model's weights, but the standard sampling and prompting mechanisms fail to surface it. Instead, the model's probability mass remains concentrated on a single "average" response, leading to what Ya‐Ling Wu calls "variance collapse"—a tendency to avoid extreme opinions and cluster around moderate responses Assessing the Human-Likeness of LLM-Driven Digital Twins in Simulating Health Care System Trust.
Toward Representative Simulation
To bridge this gap, researchers are moving beyond simple prompting. Angelina Wang and Jamie Heather Morgenstern warn that without intervention, LLMs cannot replace human participants because they fail to portray the internal diversity of identity groups Large language models cannot replace human participants because they cannot portray identity groups.
Two primary strategies have emerged to address this:
- Direct Finetuning on Distributions: Akaash Kolluri and Shengguang Wu developed the SOCRATES models by finetuning LLMs directly on millions of individual-level responses from social science experiments, significantly improving the model's ability to capture natural variance Finetuning LLMs for Human Behavior Prediction in Social Science Experiments.
- Ensemble and Persona Weighting: The Prompts to Proxies (P2P) framework, introduced by Bingchen Wang, generates a "functional basis" of diverse personas and uses regression to weight them, allowing a model to emulate a population's distribution without expensive retraining Prompts to Proxies: Emulating Human Preferences via a Compact LLM Ensemble.
Understanding why these models collapse—and how to force them to remain pluralistic—is essential if they are to ever function as reliable proxies for the complex, disagreeing, and diverse human world.