THEATRON: A Benchmark for Evaluating Persona Persistence in Long-Horizon LLM Decision-Making

Yuyan Wang, Yuechen Jiang, Zhiwei Liu, Jimin Huang, Sophia AnaniadouarXiv 2026 · arXiv

TL;DR

THEATRON is a new benchmark that moves beyond testing if an LLM can "sound" like a persona to testing if it can consistently act like one over long-term decision-making sequences. By simulating financial trading, the researchers found that while persona prompts change model behavior, they often lead to "stereotyped collapse" (repeating the same action) or "noisy switching" rather than stable, context-sensitive policies.

What Problem Does This Paper Address?

In the world of LLM personalization, we often evaluate models based on their stylistic flair—can a model talk like a pirate or a helpful assistant? However, for high-stakes applications like financial decision-support or social simulation, "persona expression" isn't enough. We need persona persistence.

The authors identify a critical gap: existing benchmarks usually focus on single-turn interactions or identity fidelity in dialogue. They don't measure whether a persona-conditioned model can maintain a coherent, distinguishable strategy over a long horizon where its previous actions affect its future state. If a model claims to be "loss-averse" but starts taking wild risks after a few bad trades, the persona has failed to persist.

Methods

The authors introduce THEATRON, a controlled benchmark using a "financial replay" task.

Figure 1: Overview of the THEATRON benchmark pipeline.
  1. The Task: Models act as trading agents over a 102-day market stream. At each step, they receive market evidence (prices, news, SEC filings) and their own account state (previous action and returns).
  2. Personas: The study uses three behavioral finance personas: Overconfidence, Loss Aversion, and Herding.
  3. Injection Methods: They test three standard ways to "nudge" the model: Zero-Shot (system prompt only), One-Shot (prompt + one example), and Chain-of-Thought (prompting the model to reason before acting).
  4. Evaluation Metrics:
    • Within-Persona Stability: Do repeated runs of the same persona produce similar results?
  5. Cross-Persona Separability: Are the trajectories of different personas actually different from each other?
  6. B/W Ratio: A ratio of between-persona distance to within-persona distance.
  7. Action Entropy: A measure of how diverse the model's actions are, used to detect "action collapse."

Datasets

THEATRON utilizes the FinAI dataset, specifically focusing on three equities:

  • AAPL (Apple): A stable, large-cap asset.
  • TSLA (Tesla): A high-volatility, sentiment-sensitive asset.
  • BMRN (BioMarin Pharmaceutical): Used for cross-sector replication.

The benchmark covers 102 trading days (Jan–May 2025) and includes adjusted closing prices, financial news summaries, and SEC filing passages (10-K/10-Q).

Key Insights and Ideas

  • Separation $\neq$ Persistence: A model might produce very different results for "Overconfident" vs. "Loss Averse" prompts, but that doesn't mean it's following a coherent policy. The authors found that high separation often comes from stereotyped policy reduction—where the model simply maps a persona to a single repeated action (e.g., always BUY) regardless of market conditions.
  • The "Noisy Switching" Failure: Conversely, some models show high action diversity but fail to maintain a stable identity across different runs of the same scenario. They react to local market cues but lack "temporal state binding."
  • Prompting Scaffolding is a Double-Edged Sword: While One-Shot and Chain-of-Thought (CoT) prompting significantly change behavior, they don't consistently fix stability issues. In some cases, they even exacerbate the collapse into deterministic action priors.
  • Comparison to Existing Benchmarks: As shown in the table below, THEATRON is unique in its support for sequential decisions combined with rigorous stability and separability metrics.
Comparison of THEATRON against other representative benchmarks across dataset types and evaluation dimensions.

Key Results

The study evaluated four major models: GPT-5.4-mini, Gemini-2.5-Flash, Qwen3-8B, and Qwen3-14B.

Main results table showing final cumulative log returns and B/W ratios across models and personas.
  • Behavioral Shifts: Persona conditioning is consequential. For example, the no-persona GPT policy chose "HOLD" 90% of the time on AAPL, but the "Overconfidence" persona increased non-HOLD actions by up to 96.5 percentage points.
  • Gemini's Collapse: Gemini-2.5-Flash under the "Overconfidence" persona repeatedly collapsed to a "BUY" action prior on TSLA, resulting in zero return variance and zero action entropy across all five test runs.
  • GPT's Instability: GPT-5.4-mini showed the opposite; under "Overconfidence," its final returns on AAPL ranged from -27.26% to +11.73% across identical runs, indicating a lack of stable policy identity.

Limitations and Future Work

The authors acknowledge several limitations:

  • Simplified Trading: The task uses a basic BUY/HOLD/SELL setup and ignores position sizing, transaction costs, and liquidity.
  • Limited Asset Scope: While they tested multiple stocks, the evaluation is still centered on a specific financial window.
  • SFT Failure: A small-scale Supervised Fine-Tuning (SFT) experiment on Qwen3-8B failed to resolve the stability-separability tension, suggesting that simple imitation learning isn't enough to instill persistent personas.

Future Work: The authors suggest that future research should focus on optimizing persona persistence at the trajectory level, ensuring models are both context-sensitive and behaviorally consistent over time.

Summary updated

Abstract

Existing evaluations often test persona expression rather than whether persona information persists in extended decision-making. We introduce THEATRON, a controlled benchmark that uses financial replay to evaluate persona persistence in long-horizon sequential decision-making. In these environments, models conditioned on fixed behavioral-finance personas observe shared daily market evidence and trajectory-specific account-state feedback and make sequential decisions over multi-month trajectories. We evaluate Zero-Shot, One-Shot, and Chain-of-Thought persona injection across four LLMs, using cumulative-return and action trajectories to diagnose within-persona stability, cross-persona separability, and action collapse. Persona conditioning produces systematic behavioral shifts beyond a no-persona baseline, and persona-level separation generally exceeds the variation induced by semantically equivalent descriptions. Nevertheless, these shifts do not reliably form coherent persona-specific policies: apparently stable separation can arise from stereotyped action collapse, while diverse policies can exhibit noisy trajectory switching. These findings expose a gap between observable persona-induced behavior and persistent, context-sensitive persona-conditioned decision policies. Code and benchmark data are publicly available at: https://github.com/yuyan12138/theatron
BibTeX
@misc{wang2026theatron,
  author = {Yuyan Wang and Yuechen Jiang and Zhiwei Liu and Jimin Huang and Sophia Ananiadou},
  title = {{THEATRON: A Benchmark for Evaluating Persona Persistence in Long-Horizon LLM Decision-Making}},
  year = {2026},
  eprint = {2611.11487},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11487},
}
Figure