What Limits LLM-based Human Simulation: LLMs or Our Design?

Qian Wang, Jiaying Wu, Zhenheng Tang, Bingqiao Luo, Nuo Chen, Wei Chen, Bingsheng He

Abstract

We argue that advancing LLM-based human simulation requires addressing both LLM's inherent limitations and simulation framework design challenges. Recent studies have revealed significant gaps between LLM-based human simulations and real-world observations, highlighting these dual challenges. To address these gaps, we present a comprehensive analysis of LLM limitations and our design issues, proposing targeted solutions for both aspects. Furthermore, we explore future directions that address both challenges simultaneously, particularly in data collection, LLM generation, and evaluation. To support further research in this field, we provide a curated collection of LLM-based human simulation resources.

Introduction

Researchers are currently in a "gold rush" to use Large Language Model (LLM) agents as proxies for humans in fields like economics, sociology, and psychology. The promise is immense: the ability to test policy changes or economic theories in "virtual towns" before real-world application. However, a fundamental issue has emerged: these simulations often fail to behave like humans. Whether it is a trading bot failing to show human-like loss aversion or a persona that loses its consistency over time, the discrepancies are significant. This paper investigates whether the primary bottleneck is the LLM itself or the design of the simulation frameworks.

Quantifying the Reliability Gap

Empirical evidence across multiple domains—social, economic, policy, and psychological—reveals a persistent "reliability gap." LLMs often exhibit "hyper-rationality" or cognitive mismatches that diverge from human patterns. For example, GPT-4’s human-likeness rating has been shown to plummet from 51% to just 13.3% after 20 turns of interaction, indicating a fragile adherence to human-like behavior.

Table 1: Empirical evidence of the performance gap between LLMs and humans across various simulation tasks.

As detailed in the table above, LLMs significantly underperform in specialized tasks. In trading simulations, GPT-4 achieved a 28% return compared to 39% for human traders. Perhaps more critically, in adversarial contexts, models like GPT-4-Turbo proved far more susceptible to attacks (27% success rate) than humans (6%), suggesting a lack of human-like skepticism and common-sense defenses.

The Dual Challenge: Model Limitations vs. Design Flaws

The failure of current simulations stems from a "double-whammy" of issues:

  1. Inherent Model Limitations: LLMs suffer from memory decay, inherent biases, and a "memory wall" where they struggle with long-term behavioral consistency. They may value immediate rewards too highly (stronger time discounting) and show weaker loss aversion than real people.
  2. Flawed Simulation Design: Even a perfect model would fail in a poorly designed simulation. Current frameworks often oversimplify human complexity, reducing rich emotional and cognitive states to simple numerical scales or basic categories, failing to capture the subtle interplay of factors driving real-world decisions.

Toward a Reliable Simulation Framework

To bridge the gap between silicon and soul, we propose a systematic framework for "Reliable LLM-based Human Simulation." This approach moves beyond simple prompting toward a multi-layered architecture focused on data foundations, model enhancement, and rigorous validation.

Figure 1: The proposed framework for reliable human simulation, integrating richer data foundations, bias mitigation, and multi-level validation loops.

Key components of this framework include:

  • Data Foundations: Moving toward multi-modal data collection (including physiological and cognitive markers) to ground agents in "lived experience."
  • LLM Enhancements: Implementing reflection mechanisms, layered memory systems, and targeted debiasing techniques to maintain cognitive consistency.
  • Systematic Validation: Shifting from simple accuracy metrics to multi-level validation that monitors individual, group, and systemic behavior simultaneously.

Conclusion

Advancing LLM-based human simulation requires a paradigm shift. We must address both the silicon limitations of the models and the oversimplified designs of our frameworks. While moving toward multi-modal, sensor-grounded simulations introduces higher costs and ethical complexities regarding privacy, it is a necessary step for creating agents that truly reflect the stochastic and context-dependent nature of human behavior. Until such systematic validation becomes standard, LLM simulations should be treated as experimental tools rather than definitive predictors of human society.

Summary updated

Abstract

We argue that advancing LLM-based human simulation requires addressing both LLM's inherent limitations and simulation framework design challenges. Recent studies have revealed significant gaps between LLM-based human simulations and real-world observations, highlighting these dual challenges. To address these gaps, we present a comprehensive analysis of LLM limitations and our design issues, proposing targeted solutions for both aspects. Furthermore, we explore future directions that address both challenges simultaneously, particularly in data collection, LLM generation, and evaluation. To support further research in this field, we provide a curated collection of LLM-based human simulation resources.\footnote{https://github.com/Persdre/llm-human-simulation}
BibTeX
@misc{wang2025what,
  author = {Qian Wang and Jiaying Wu and Zhenheng Tang and Bingqiao Luo and Nuo Chen and Wei Chen and Bingsheng He},
  title = {{What Limits LLM-based Human Simulation: LLMs or Our Design?}},
  year = {2025},
  eprint = {2501.08579},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.08579},
}
Figure