LLM social simulations hand each agent a human identity ("a 67-year-old retired farmer, Republican, low trust in institutions") and ask it to decide something: whether to buy a product, how hateful a post is, what number to estimate after hearing peers. They also ask it to explain why. Validation almost always checks the decisions against human data. Two recent workshop papers suggest this check is too weak. In one, LLMs imitating superforecasters produced forecasts and rationales that looked right, but the reasoning patterns inside those rationales differed from the humans' (Can LLMs Faithfully Enact Superforecaster Personas?). In the other, LLM-simulated reasons for a purchase decision mostly repeated the product description instead of recovering why a given respondent accepted or rejected it (Reason-Mediated Behavioral Models for Auditing LLM Social Simulators). If an agent's identity changes its answer directly while its stated reasons are written afterward, the simulation can match human outcomes today. It then fails exactly where simulations are supposed to help: under interventions that work by changing people's reasons, such as counter-arguments, nudges, and new framings.
We propose a direct test of this and an architecture that fixes the problem when it appears. The test is a reason transplant. Keep an agent's persona fixed, replace its generated reasons with reasons generated under a different persona, and measure how far its decision moves. If the decision barely moves, the reasons were decorative. The fix is a reason bottleneck. The agent writes structured, signed reasons from its persona, and a separate step that never sees the persona makes the decision from those reasons alone. All of the identity's influence must then pass through reasons that can be inspected and compared with what real people from that group say.
Core Claim
Our hypothesis has two parts. First, in current persona-prompted agents, most of the effect of identity on decisions does not run through the stated reasons. Second, this hidden shortcut explains a puzzle that three independent studies report: step-by-step reasoning makes persona simulation worse. Chain-of-thought reasoning (prompting the model to write intermediate steps before answering) reduced partisan bias and human-likeness in deliberating agent groups (The Wisdom of Partisan Crowds). It degraded preference alignment for census-derived personas (PERSONA). It produced near-identical rationales across 21 demographic personas, with Krippendorff's $\alpha \geq 0.80$ (Persona Prompting as a Lens on LLM Social Reasoning).
Our account is that once the model writes out reasoning, its decision partly follows that reasoning. Because the reasoning comes from a persona-generic template, it washes out the identity signal. Prompting cannot hide the problem under a bottleneck. The design therefore separates two questions that are currently confounded: whether the agent has the right reasons, and whether it acts on them. The contribution is new because prior persona work measures outcome fidelity or rationale similarity, never whether the rationale causally carries the persona's effect. Prior chain-of-thought faithfulness work never conditions on identity. Measuring Faithfulness in Chain-of-Thought Reasoning truncates and corrupts reasoning on multiple-choice reasoning suites such as AQuA and ARC. FRIT and Counterfactual Simulation Training train for faithfulness on math, logic and ethics benchmarks.
Research Program
The central quantity is a mediation share. Let $Y(d, z)$ be the agent's decision when its persona prompt is $d$ and its reasoning text is $z$, and let $z_d$ be the reasons the agent generates under persona $d$. For a pair of personas $d, d'$ we define
$$M(d \to d') = \frac{Y(d, z_{d'}) - Y(d, z_d)}{Y(d', z_{d'}) - Y(d, z_d)},$$
which is the fraction of the total persona effect on the decision recovered by transplanting only the reasons. A value of $M$ near 1 means identity acts through reasons. A value near 0 means the reasons are post-hoc. We average over persona pairs whose total effect is reliably nonzero, to avoid dividing by noise.
Transplants are impossible on humans, so we use a second measure that has a human counterpart, following the logic of Reason-Mediated Behavioral Models. On human data, fit a model mapping descriptors $D$, stimulus $X$ and signed reason state $Z$ to outcome $Y$. Then ask whether each LLM agent's $(Z, Y)$ pairs are coherent under this human mapping, meaning whether the agent decides the way a human holding those reasons would. We also report an echo rate: the fraction of generated reason codes that paraphrase the stimulus text rather than reflect the respondent. This is the failure the seed audit describes.
The work proceeds in three stages. The first stage is an audit. We compute mediation share, reason–outcome coherence, echo rate, and reason-state agreement with humans for standard persona prompting, few-shot persona prompting, attribute-summarization prompting, and chain-of-thought prompting. We use three human datasets that pair identity, outcome and stated reasons. The first is the sunscreen concept test from the seed audit (94 respondents, three concepts each, open-ended rationales coded into signed reasons). We will request it from the authors or rerun the same design with a larger panel if unavailable. The second is the superforecaster rationale data from the seed persona paper, obtained the same way. The third is HateXplain and BRWRR, where annotator labels come with word-level rationales (HateXplain).
The second stage is the reason bottleneck. A persona-conditioned generator writes signed reasons in the codebook of each dataset, plus free text. A persona-blind decider then maps those reasons to the outcome. We test two variants of the generator. One is prompted with same-group human reason exemplars. The other is additionally trained with a FRIT-style preference objective that prefers reason sets whose transplant moves the blind decider in the direction human data says. The FRIT recipe, preference optimization over causally important reasoning steps, is reused here with identity-conditioned targets instead of math answers.
The third stage is deliberation. We place bottlenecked agents in the three-round partisan estimation protocol of The Wisdom of Partisan Crowds, built on Becker et al. data. We test whether restoring persona-specific reasons recovers the partisan bias and crowd-wisdom effects that chain-of-thought destroyed.
Baselines
The naive reference is the generic persona prompt answering directly with no reasoning. Prior work suggests it is surprisingly strong on outcomes. It is run with the same models, the same persona strings and the same number of samples per respondent. Its only weakness is that it produces no reasons to audit. Beating it means matching its outcome fidelity, within confidence intervals on held-out respondents, while also producing reasons with high mediation share.
The strongest prompting competitor is attribute summarization before answering, which outperformed step-by-step reasoning in PERSONA. The file reports Cohen's $\kappa$ of 0.6–0.8 for its human validation but no comparable number on our tasks. It is the most plausible cheap fix for the chain-of-thought penalty, because it front-loads identity without a generic reasoning template. We run it with the same exemplars and token budget as the bottleneck. Beating it means higher mediation share and coherence at equal or better outcome fidelity.
The strongest learned competitor is fine-tuning on human outcomes. In The Wisdom of Partisan Crowds, this raised the Human Likeness Index from −33.95 to 50.11 on the training set. The same paper reports HLI 12.82, $\Delta\epsilon = -7.59$ and partisan bias 5.23 for ChatGPT with detailed personas and no chain-of-thought, which is the prompting number our deliberation stage must reach or exceed. We fine-tune the same base model on the same training respondents and give it the same gradient-step budget as our preference-trained generator. On HateXplain, the reference supervised classifier is BERT-HateXplain at macro F1 0.687 (HateXplain). We report it only to anchor label quality, not as a persona method. For the sunscreen and superforecaster data, no reported numerical result was found on Lacuna for either task. Baselines there are computed fresh.
The key ablation removes the bottleneck but keeps everything else: the same generator writes the same structured reasons, but the decider also sees the persona. If mediation share and human coherence drop when the persona leaks to the decider, the gains come from forcing identity through reasons and not from better reasons alone. A second ablation keeps the bottleneck but uses a persona-free reason generator. It should erase group differences in outcomes, which confirms that the bottleneck carries persona signal rather than inventing it.
Why not just…?
Why not just drop reasoning and let personas answer directly, since that already works better? For aggregate outcomes it may well be enough, and we report it as the reference. But a direct answer cannot be checked for mechanism. As the seed audit argues, a simulator that gets outcomes right for the wrong reasons will mispredict interventions that act on reasons. Our deliberation stage tests this. If direct-answer agents match the bottleneck under counter-argument interventions as well as at baseline, this proposal's practical motivation fails, and we will say so.
Why not just summarize persona attributes first, as PERSONA recommends? Summarization explains why chain-of-thought hurts less, but nothing in PERSONA shows that the resulting decisions run through persona-appropriate reasons. The audit stage measures its mediation share directly. If it is already high, the bottleneck adds little and we will recommend summarization instead. That outcome would be a useful negative result.
Why not just fine-tune on human behavior? Fine-tuning on outcomes rewards the shortcut: it teaches the model to map identity to answer more accurately, with no pressure on reasons. The Wisdom of Partisan Crowds also reports that its fine-tuned models overfit and produced extreme values that needed filtering. We predict outcome-fine-tuned agents will score well on outcome fidelity and worst on mediation share. If their reasons turn out to be coherent under the human mapping anyway, this alternative is a real threat, and the cheaper method wins.
Why not just sample reasons from the empirical human distribution for each group? Within a fixed stimulus set this is likely the best reason generator possible, and we include it as an oracle-like baseline feeding the same blind decider. It cannot produce reasons for stimuli no human has seen, which is the situation simulations are built for. We therefore evaluate on held-out concepts, held-out forecasting questions and held-out posts. If empirical sampling from the nearest seen stimuli still beats LLM-generated reasons there, the right conclusion is that LLMs should decide but not reason. We would report that.
Proposed Experiments
We run four models spanning sizes, including at least one open 7–8B and one 70B-class model, because The Power of Simplicity found that iterative reasoning hurt Llama-3.1-8B but helped larger models, and the mediation story may differ by scale. Splits hold out both respondents and stimuli.
The first decisive check is the audit prediction. Mediation share for standard and chain-of-thought persona prompting should fall well below the human reasons' incremental predictive value. Echo rate should be high on the sunscreen data, replicating the seed audit quantitatively. If mediation is already high, the core claim is wrong.
The second check is the chain-of-thought explanation. Under chain-of-thought, between-persona divergence of reasons should be low, consistent with the $\alpha \geq 0.80$ agreement in Persona Prompting as a Lens, while mediation share stays moderate. Low divergence combined with real mediation is the mechanism by which reasoning would wash out identity.
The third check is the deliberation test. Bottlenecked agents should recover partisan bias and the crowd-wisdom effect at or above HLI 12.82, $\Delta\epsilon = -7.59$, and $BPB = 5.23$. In an added condition, peers exchange short reason statements rather than only numbers. Here we measure whether agents update on the content of opposing reasons. We do this by transplanting a peer's argument while holding the peer's number fixed.
The final check is whether the preference-trained generator improves reason-state agreement with held-out humans without lowering outcome fidelity. A positive answer would show that identity-conditioned agents can be made to act on reasons people actually hold. A negative answer would show that, at present, persona rationales in social simulation should be treated as commentary, not mechanism.
Workshop authors to connect with
- Can LLMs Faithfully Enact Superforecaster Personas?: Andrew Robert Williams, Evan Jiang, Martin Weiss, Nasim Rahaman, Hugo Larochelle
- Reason-Mediated Behavioral Models for Auditing LLM Social Simulators: Atharva Pandey, Gautam Jajoo
References
- Can LLMs Faithfully Enact Superforecaster Personas?
- Reason-Mediated Behavioral Models for Auditing LLM Social Simulators
- The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents
- PERSONA: A Reproducible Testbed for Pluralistic Alignment
- Persona Prompting as a Lens on LLM Social Reasoning
- Measuring Faithfulness in Chain-of-Thought Reasoning
- FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulness
- Counterfactual Simulation Training for Chain-of-Thought Faithfulness
- HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection
- The Power of Simplicity in LLM-Based Event Forecasting