Does the Persona Act Through the Reasoning? Mediation Tests for Persona-Conditioned Game Agents

Social simulations increasingly give a language model a character, such as a cautious retiree, an aggressive trader or a partisan voter, and then read the model's stated reasoning as evidence of how that character thinks. This proposal asks whether that reading is justified. When a persona changes what an agent does, does the change travel through the reasoning the agent writes down, or does the persona push the action directly while the written reasoning is decoration added afterward? We propose to answer this causally. In a poker environment where every decision can be checked against a computable reference, we give each persona an explicit, machine-checkable decision rule. We then swap persona prompts and reasoning traces independently, which lets us measure what share of the persona's effect on actions is carried by the stated reasoning.

The stakes are practical. Can LLMs Faithfully Enact Superforecaster Personas? found that persona-like rationales can match human style while differing in their underlying reasoning patterns. Can LLMs Simulate Personas with Reversed Performance? found that OpenAI o1 kept 99% accuracy on GSM8k when told to play a struggling student, imitating the hesitant style without changing the substance. If persona rationales in agent simulations work the same way, then the "linguistic justifications" that simulation papers report, for example in Decoding Echo Chambers, describe something other than the policy that produced the behavior.

Core Claim

We claim that persona faithfulness in agents should be measured as a mediation quantity rather than as textual consistency. Mediation here means the fraction of a persona's causal effect on actions that flows through the agent's written reasoning. We also predict that this fraction is lowest when the persona conflicts with the base model's default disposition.

The conflict prediction comes from two findings. Doing What They Say, Not What They Reason found that Claude Haiku 4.5's poker rule misapplications are overwhelmingly risk-averse. Persona Prompting as a Lens on LLM Social Reasoning found that reasoning templates resist persona steering, with inter-persona agreement measured as Krippendorff's α ≥ 0.80. Together these suggest that an aggressive persona will produce aggressive-sounding reasoning while the action drifts back toward the cautious default, or alternatively that the persona will move actions through a channel the reasoning never mentions. Our design is built to tell these two failure modes apart.

The measurement works as follows. Write $m(x, y)$ for the fraction of decisions whose executed action matches the reference policy of a target persona $p$, when the system prompt is persona $x$ and the reasoning trace is the one generated under persona $y$ (prefilled into the model's response). The neutral persona is denoted $0$. The total effect of the persona is $m(p,p) - m(0,0)$. The effect carried by the reasoning is $m(0,p) - m(0,0)$: we keep the neutral prompt and insert only the persona's reasoning. The mediated share is

$$\rho_p = \frac{m(0,p) - m(0,0)}{m(p,p) - m(0,0)}.$$

A share $\rho_p$ near 1 means the reasoning carries the persona. A share near 0, combined with a large direct effect $m(p,0) - m(0,0)$, means the persona acts around the reasoning. A low share combined with persona-flavored text means the reasoning is a veneer.

What Prior Work Leaves Open

Three lines of work each cover part of this question.

The first line studies agent faithfulness. Doing What They Say, Not What They Reason separates the links from reasoning to stated conclusion and from stated conclusion to executed action in Texas Hold'em. It shows that the second link is mostly faithful and that the gap sits upstream. Its agents carry no persona, however, so it cannot say whether personas move or widen that gap. We reuse its harness and its insistence on machine-checkable conclusions.

The second line studies persona fidelity. Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing scores whether responses entail or contradict persona facts, using a distilled natural-language-inference (NLI) checker, and optimizes that score. Its notion of faithfulness is agreement with the profile, not a causal link between reasoning and action. Conformity, Confabulation, and Impersonation documents personas drifting in group chat, including confabulated opinions in 1.1–1.64% of reflections. It tracks opinions rather than the reasoning that produced a stance flip. Both studies motivate our work by showing that persona surfaces are unreliable, but neither tests a causal pathway.

The third line studies explanation faithfulness and supplies our method. Language Models Don't Always Say What They Think showed that biasing cues change answers without being mentioned in the rationale. RFEval injects flawed reasoning prefixes and reports 49.7% unfaithful outputs across 12 reasoning models on static question-answering tasks. On Measuring Faithfulness or Self-consistency of Natural Language Explanations warns that agreement between an explanation and its prediction is not faithfulness. Our swap design extends RFEval-style prefix intervention in two ways. First, the intervened content is a whole persona's reasoning rather than a single flawed premise. Second, the outcome is a sequential game action with a computable reference rather than a question-answering label.

Research Program

The first component turns personas into reference policies. In the poker harness, each persona receives a decision rule the harness can compute from game state. A risk-seeking persona calls when equity exceeds pot odds scaled by a factor below 1. A risk-averse persona uses a factor above 1. A "mechanical" persona uses the plain equity rule. A bluff-prone persona raises at a stated frequency in specified spots. Every persona is written in two forms. The narrative form is a character description of the kind social simulations actually use, such as "a fearless ex-trader who hates folding." The rule form states the threshold explicitly. Each persona therefore has a reference action $a^*_p$ for every decision, alongside the persona-free equity reference from the original harness.

The second component is the reasoning-swap protocol. For each sampled game state, we generate reasoning under the neutral prompt and under each persona prompt. We then prefill each reasoning trace into the response under every system prompt and record the executed action, which gives the full $m(x,y)$ grid. Prefilled traces from persona $p$ often contain self-references such as "as someone who never folds." To prevent the swap from simply leaking the persona label, we run a second variant in which a scripted rewrite removes identity language. That variant keeps only the stated quantities and the conclusion. If $\rho_p$ survives this stripping, the reasoning content is doing the work, not the persona's name.

The third component checks the reasoning itself. Following the elicitation diagnostic in Doing What They Say, Not What They Reason, each trace must state its equity estimate, its pot odds, the rule it is applying and its decision. We verify these against the simulator. This classifies each persona-inconsistent action into one of five causes:

  • an input error;
  • a borderline case;
  • application of the wrong rule, such as the base model's default instead of the persona's;
  • a correct persona rule followed by a conclusion that contradicts it;
  • a correct conclusion followed by an action that contradicts it.

The "veneer rate" is the fraction of decisions where the trace explicitly invokes the persona's rule but the action matches the neutral model's action instead of $a^*_p$.

The fourth component tests transfer to a second environment. We repeat the protocol on the Social Gym normal-form and economic games from Social Gym and SPaRTan. These games have algorithmically verifiable outcomes, and simple personas such as "self-interested" or "fairness-seeking" admit computable reference responses. The aim is to check whether a low $\rho_p$ is a property of poker or a general property of persona prompting.

Models will include the two families studied in the poker paper, Claude Haiku 4.5 and DeepSeek-Reasoner, plus open-weight models from the persona-lens study, Qwen3-32B and the GPT-OSS series. With open weights we control prefilling exactly. Some hosted reasoning models do not allow prefilling their hidden reasoning. For those we prefill the visible reasoning field and say so explicitly, because this is a real limitation on what can be claimed for them.

Baselines

The naive reference point is the neutral, persona-free agent from the original poker harness. It defines $m(0,0)$ and the base model's default action at every state. Every persona effect is measured relative to it. A persona that does not move actions beyond this point has no effect to mediate. We will report that as a finding in its own right, analogous to o1's unchanged 99% in the reversed-performance study.

The strongest existing measurement method is RFEval-style counterfactual prefix intervention RFEval, which reports 49.7% unfaithful outputs across its tested models. That number comes from static tasks, not agents. We implement it in the poker harness by injecting a single flawed premise, such as a wrong pot-odds value, into the neutral agent's trace. We use the same states, models, sampling settings and number of samples per cell as the swap protocol. Our protocol beats this baseline if it reveals failures the prefix test misses. Concretely, we look for personas whose agents pass the flawed-premise test, following the injected premise into the action, yet show low $\rho_p$. That pattern would mean the agent "follows its reasoning" in the RFEval sense while its persona acts through another channel.

The strongest competitor for persona fidelity is the APC-style NLI consistency score Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing. We approximate it with an NLI judge that scores whether each rationale entails the persona description. Its reported figure is about agreement with GPT-4, 90% logic consistency, not about action faithfulness, and we found no result for it on agent decisions in the file. We compute it on the same traces. Our measure is informative beyond it if the correlation between APC score and $\rho_p$ across personas and models is weak. In particular, we look for narrative personas that score high on APC and low on mediation.

The key ablation is the rule form versus the narrative form of each persona. Rule-form personas make the reasoning path explicit, so they should have high $\rho_p$. If narrative personas reach the same $\rho_p$, the veneer concern is unfounded for this setting.

The mechanical-instruction intervention from Doing What They Say, Not What They Reason serves as a remedy baseline. The file reports only that it "roughly halves" Haiku's rule misapplications; no exact number was found on Lacuna. Few-shot persona prompting, which Can LLMs Faithfully Enact Superforecaster Personas? found closer to human forecasts than instruction prompting, serves as a second remedy baseline.

Why not just…?

Why not just measure whether persona-conditioned actions match the persona's reference policy, and ignore the reasoning? For an engineer who only needs the right behavior, this is sufficient, and we report it as $m(p,p)$. The objection is a real threat to the motivation, but not to the measurement. Social simulations publish rationales as findings, as in the echo-chamber and superforecaster studies, and the poker paper showed that removing explicit intermediate stages hides where failures occur. If $\rho_p$ turns out high across personas and models, then action matching would have been enough, and we will say so.

Why not just score rationales for consistency with the persona using an NLI judge? Consistency is not causation. The self-consistency critique in On Measuring Faithfulness or Self-consistency and the steering paradox in Persona Prompting as a Lens both show that fluent, on-persona text can sit alongside unchanged underlying reasoning. Our baseline comparison tests this directly. If APC score predicts $\rho_p$ well, the cheaper judge should replace our protocol.

Why not just state the persona as an explicit rule, as the mechanical instruction did? This could plausibly solve the action-faithfulness problem, which is why we built it in as the rule-form ablation rather than arguing against it. The open question is whether that result transfers. Simulations use narrative characters, and the poker paper found that DeepSeek-Reasoner's errors differ in kind from Haiku's. Whether a rule-form fix transfers across model families is therefore an empirical question, which our model-by-form grid answers.

Proposed Experiments

The primary experiment samples a few thousand decision states from the poker harness. For each state it fills the full $m(x,y)$ grid for five personas, each in narrative and rule form, across four models. It reports $\rho_p$ with bootstrap intervals over states.

The first decisive check is the conflict prediction. Personas opposed to the base model's measured default, such as risk-seeking personas for a model whose neutral errors are risk-averse, should show lower $\rho_p$ and higher veneer rates than personas aligned with that default. A flat pattern would falsify the claim.

The second decisive check is the stripped-swap control. If $\rho_p$ collapses once identity language is removed, then reasoning "mediates" only by restating the label. That outcome is itself evidence of veneer, and we will report it as such.

The third check is the error decomposition. For persona-inconsistent actions, we determine whether failures concentrate in wrong-rule application, the upstream stage where the poker paper located the gap, or whether personas create new failures between conclusion and action. Personas creating such new failures would overturn that paper's finding that executed actions follow stated conclusions.

The transfer experiment repeats the grid on Social Gym economic and normal-form games, with 30 games per condition to match that benchmark's evaluation size. If the ordering of personas by $\rho_p$ is preserved across environments, the measure reflects a property of persona prompting rather than of poker.

Workshop authors to connect with