Position: LLM Social-Simulation Agents in the Wild Cannot Serve as Social Scientific Evidence Without an Identification Strategy

Zezheng Lin, Fengming LiuAIWILD @ ICML 2026 · OpenReview

TL;DR

This position paper argues that while Large Language Models (LLMs) can impressively mimic human social behavior, they cannot yet serve as valid scientific evidence for social science. The authors contend that high "predictive fidelity" (matching human results) is often a mirage caused by the model simply retrieving training data or following prompt instructions, rather than genuinely simulating human social dynamics.

What Problem Does This Paper Address?

In the "post-GPT-4" era, a new research program has emerged: using LLMs as "silicon subjects" to replace humans in social science experiments. Some studies show that GPT-4’s responses correlate strongly ($r=0.85$) with human treatment effects across hundreds of experiments.

However, the authors identify a critical gap borrowed from econometrics: the distinction between prediction and identification. A model can be highly predictive (it gets the "right" answer) without identifying the causal mechanism (the "why" behind the answer). If an LLM "replicates" a famous psychology study, is it because it understands human nature, or because that study was in its training data? Without an "identification strategy" to separate these possibilities, LLM simulations remain "agentic outputs masquerading as human-subject evidence," which could lead to flawed policy decisions if used in the real world.

Methods

The authors formalize the problem by decomposing an LLM agent's behavior ($Y_i$) into three distinct mechanisms:

  1. $f_{train}$ (Training-Prior Retrieval): The model retrieves statistical regularities it learned during pre-training (e.g., it "knows" how people usually answer the Ultimatum Game because it read about it).
  2. $g_{prompt}$ (Role-Script Compliance): The model follows the specific instructions in the prompt (e.g., "Act like a conservative voter").
  3. $h_{emerge}$ (Interactional Emergence): The "holy grail" where complex social patterns arise naturally from agent interactions.

The authors argue these three are observationally equivalent: you can't tell which one produced the result just by looking at the final data. To solve this, they propose a six-item Identification Checklist for researchers, including prompt ablation, placebo tests (e.g., shuffling agent demographics), and training-data leakage audits.

Datasets

As a position paper and audit, this work evaluates the datasets and results of several landmark papers in the field, including:

  • Generative Agents (Park et al., 2023): The "Smallville" sandbox of 25 agents.
  • Homo Silicus (Horton, 2023): Replications of classic behavioral economics (Kahneman, Tversky, etc.).
  • Silicon Sampling (Argyle et al., 2023): Using ANES (American National Election Studies) data to condition LLMs.
  • The Hewitt Result (Hewitt et al., 2024): A massive archive of 70 pre-registered survey experiments with 105,165 human participants.

Key Insights and Ideas

  • The "Hyper-Accuracy" Trap: In some simulations, LLMs are more accurate than human crowds. This is actually a red flag for retrieval ($f_{train}$); the model is accessing the "ground truth" from its training data rather than simulating the messy, biased way humans actually think.
  • Theory-Consistent Priors: Even for studies published after a model's training cutoff, LLMs can "predict" results because they have learned the underlying social theories (e.g., "nudges work this way") that researchers use to design those experiments.
  • The Aggregation Problem: A high correlation across 476 different effects (the $r=0.85$ result) can mask the fact that the model fails miserably at simulating the internal logic or variance of any single experiment.
  • Substitution vs. Forecasting: LLMs are currently great for forecasting (predicting what might happen based on past data) but dangerous for substitution (replacing humans to discover new social truths).

Key Results

The authors audited the "pro-simulation" literature and found that none of the major papers currently provide an identification strategy that rules out simple retrieval or prompt-following.

  • They highlight that while "pro-simulation" papers emphasize high-level correlations, a parallel "critical wing" of research finds that LLM responses are highly sensitive to minor prompt changes, exhibit "machine bias," and drift significantly over time—all signs that the underlying social mechanism hasn't been "identified."
  • The paper concludes that until the proposed checklist is adopted, LLM simulation results should be treated as hypotheses or forecasts, not as causal evidence.

Limitations and Future Work

The authors acknowledge that their proposed checklist increases the cost and complexity of LLM research. However, they argue this cost is small compared to the cost of human-subject trials.

Future Work: The authors call on major AI conferences (like NeurIPS) to adopt "identification-strategy disclosure" as a requirement for publishing LLM-based social science. They provide a "worked example" in the appendix of how a truly "identified" multi-agent study would look, using novel coordination games with "nonsense" labels to prevent the model from relying on its training data.

Summary updated

Abstract

This position paper argues that LLM social simulations cannot substitute for human-subject evidence without an identification strategy. Recent work shows high predictive fidelity --- GPT-4 simulations correlate strongly with human treatment effects across hundreds of contrasts, including a post-training-cutoff subset --- while critical work shows synthetic respondents fail regression, prompt-sensitivity, and temporal-stability tests. These findings are not contradictory: prediction asks whether outputs match observed outcomes, while substitution asks whether the simulation identifies the social data-generating process. We formalize the problem as observational equivalence among three mechanisms: training-prior retrieval, prompt-induced role compliance, and genuine interactional emergence. The same outcome distribution can be rationalized by multiple combinations of these mechanisms, so predictive fit alone cannot identify emergence. We audit the principal LLM social-simulation literature through this lens, concede the strongest predictive-fidelity result, and show why it cannot license replacing respondents or experiments. NeurIPS should require an identification standard: simulations may generate hypotheses or forecasts, but they become evidence only when their identifying assumptions are explicit, testable, and stress-tested. Positioned for AIWILD, the paper treats LLM social-simulation pipelines as agentic AI deployed in the wild of social-scientific decision-making, where reliability and trustworthiness require not only predictive fit but identification of the agentic data-generating process; without it, agentic outputs masquerade as human-subject evidence and propagate into policy decisions through a failure mode that is invisible to standard agent benchmarks.
BibTeX
@inproceedings{lin2026position,
  author = {Zezheng Lin and Fengming Liu},
  title = {{Position: LLM Social-Simulation Agents in the Wild Cannot Serve as Social Scientific Evidence Without an Identification Strategy}},
  booktitle = {AIWILD @ ICML},
  year = {2026},
  url = {https://openreview.net/forum?id=CXY3MuWDbm},
}
Figure