Social science has long faced a fundamental bottleneck: human participants are expensive, difficult to recruit at scale, and constrained by ethical boundaries. The rise of Large Language Models (LLMs) offers a tantalizing alternative. What if we could instantiate thousands of diverse, digital personas in seconds to simulate market competition, psychological dynamics, or political deliberation?
This vision of "silico-human" simulation promises a paradigm shift in how we study society. However, treating an LLM as a "digital twin" of a human participant introduces a profound methodological challenge. LLMs are trained to predict text, not to possess human psychology. To use them reliably, researchers must answer a critical question: Do LLM agents actually think and act like us, or are their simulated societies merely an illusion?
The Strategic Mismatch: Hyper-Rationality and Bounded Logic
When we deploy LLMs in behavioral economics or game theory frameworks, they often mimic human behavior on the surface. For instance, David Jurgens et al. and Chengxing Xie et al. demonstrated that advanced models like GPT-4 can exhibit human-like trust, reciprocity, and risk perception when placed in standard economic games. Similarly, in competitive market simulations, Qinlin Zhao et al. found that agents naturally replicate complex economic phenomena like niche differentiation and the "rich-get-richer" Matthew Effect.
However, looking closer at how these decisions are made reveals a stark divergence from human psychology. Humans operate under bounded rationality—we use heuristics, make emotional compromises, and satisfice. LLMs, by contrast, can be jarringly hyper-strategic.
Using an automated program discovery tool, Caroline Wang et al. discovered that frontier LLMs do not act like average human players; instead, they employ highly complex internal models designed to aggressively exploit their opponents.
[Human Strategic Behavior] ───(Bounded Rationality, Heuristics)───> Bounded Outcomes
[Frontier LLM Behavior] ───(Hyper-Optimization, Exploitation)──> Hyper-Strategic Outcomes
Paradoxically, this hyper-strategic nature coexists with fundamental logical failures. Caoyun Fan et al. systematically analyzed LLMs across classical game-theoretic scenarios and found that the models struggle with uncommon preferences and fail to refine their beliefs based on complex patterns. Most strikingly, the authors observed that LLMs frequently "overlook" their own logical deductions when it comes time to take an action, breaking the alignment between desire, belief, and behavior that characterizes human rationality.
The Alignment-Simulation Tradeoff
Why do LLMs deviate so sharply from human populations? A major culprit is the way modern models are trained.
To make LLMs safe and useful, creators use instruction tuning and Reinforcement Learning from Human Feedback (RLHF). This process aligns the model to be a polite, helpful, and safe assistant. But as Tiancheng Hu et al. demonstrate through their benchmark SimBench, this creates a fundamental alignment-simulation tradeoff.
Instruction Tuning / RLHF
│
├───> High Alignment (Helpful, Safe, Predictable Assistant)
└───> Low Simulation Fidelity (Loss of High-Entropy Human Opinions)
Real human populations are messy, fragmented, and hold conflicting, "high-entropy" opinions. Instruction tuning systematically crushes this entropy, forcing the model toward safe, homogenized responses. As a result, when researchers try to simulate a representative sample of humans, the models struggle to capture true demographic diversity.
Furthermore, as highlighted by Jacy Reese Anthis et al., aligned models suffer from systematic flaws like sycophancy (agreeing with the user rather than acting authentically) and alienness (exhibiting behaviors completely foreign to human nature). This makes them easily manipulable; for example, the trust behaviors observed by David Jurgens et al. were far easier to disrupt through negative manipulation than real human trust dynamics would be.
Methodological Artifacts vs. True Emergence
Because LLMs are highly sensitive to prompting, many celebrated examples of "emergent social behaviors" in multi-agent systems are actually illusions caused by experimental design.
To address this, Jiaxu Zhou et al. introduced the PIMMUR principles, a framework designed to ensure methodological rigor in collective behavior simulations. When evaluating the existing literature, the authors found a sobering reality: nearly 90% of published LLM-based social simulations violate at least one core principle. Many "emergent" behaviors are not authentic social dynamics at all, but rather artifacts of instruction-following or model-specific biases.
Without grounding agent motivations in established psychological theory, simulations default to flat, unrealistic caricatures. For example, in educational simulations tracking complex behaviors like bullying, Jingzhe Lin et al. note a pervasive "Fidelity Challenge," where agents lack genuine psychological needs and default to simplistic actions that fail to reflect real human social orientations.
Mapping the Psychological Fingerprint
To move past fragmented, ad-hoc validations, the field is shifting toward standardized, multi-dimensional benchmarking of agent behavior.
Instead of asking if a model can pass a specific test, researchers are mapping the broader psychological fingerprint of LLMs. Jen-tse Huang et al. argue that we must evaluate the holistic humanity of these models—their personalities, shifting motivations, and emotional capacities—before trusting them as experimental proxies.
This requires moving away from static, multiple-choice benchmarks, which can underestimate complex behaviors. For instance, Yang Xu et al. showed that static tests fail to capture long-horizon behaviors like "chains of deception," where one small lie under pressure necessitates an escalating series of fabrications over time.
To capture this fluidity, Yuxuan Lei et al. proposed a computational framework that leverages large-scale web data to evaluate open-ended human behaviors across three core axes:
- Rationality: Does the agent's action logically follow from its environment and goals?
- Consistency: Does the agent maintain its persona and choices across identical contexts?
- Diversity: Does the aggregate behavior of the agents match the statistical distribution of real human populations?
By evaluating models along these rigorous dimensions, researchers are beginning to chart exactly where silico-human simulations succeed—such as replicating collective intelligence and belief convergence in partisan deliberation (Yun-Shiuan Chuang et al.)—and where they fail, such as capturing the nuanced professional standards and trust thresholds of human experts in collaborative environments (Aayam Bansal et al.).
Ultimately, LLMs remain a deeply promising tool for the social sciences. However, they are not ready-made drop-in replacements for human subjects. Unlocking their potential requires treating simulation fidelity not as an assumption, but as a variable that must be continuously measured, benchmarked, and verified.
Key Foundational Work
Can Large Language Model Agents Simulate Human Trust Behavior? (2024) — contributed 5 concept elements to this direction.
The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents (2024) — contributed 2 concept elements to this direction.
Position: LLM Social Simulations Are a Promising Research Method (2025) — contributed 2 concept elements to this direction.