TL;DR
PsyWorld introduces a "profile-conditioned world model" that predicts human behavior by learning how stable psychometric traits interact with dynamic environment states. By using realized state transitions as a source of supervision during training, the model achieves significantly higher accuracy and more human-like behavioral distributions when generalizing to previously unseen individuals.
What Problem Does This Paper Address?
Predicting human behavior is difficult because it is driven by two distinct factors: stable traits (like personality, motivation, or stress tolerance) and time-varying states (the immediate task context or environment).
Existing AI agents typically treat personality as a static "prompt" or a direct input to a policy. This approach often fails to capture the nuance of how a person’s traits influence their reaction to changes in the environment. For example, a highly stressed individual might react differently to a task failure than someone with high resilience. The authors argue that current models leave valuable information on the table by ignoring "realized state transitions"—the actual way the environment and behavior change over time—as an explicit source of training supervision.
The central challenge is to exploit these realized changes during training without "cheating" at inference time, where the future state is unknown.
Methods
The authors propose PsyWorld, a hierarchical framework that separates stable profiles from dynamic states. Its core is a profile-conditioned state world model.
Key components include:
- Context Encoder: Fuses the stable psychometric profile ($q_i$), current environment state ($s_t$), and current macro-behavior ($a_t$) into a context vector ($h_t$).
- Prior-Posterior Scheme: During training, a Posterior Encoder looks at the actual next state change ($\Delta s_t$) and maps it to a discrete code from a learned codebook (Vector Quantization). Simultaneously, a Learned Prior tries to predict this same code using only the current context. At inference (rollout), only the Prior is used, ensuring no future information leaks into the prediction.
- State Decoder: Predicts the next environment state ($\hat{s}_{t+1}$) conditioned on the transition code.
- Gated Macro Decoder: Predicts the next high-level action. It uses a "gated" architecture that decides whether to rely on the direct context or the transition-informed latent branch.
- Simulation-Grounded Execution: The predicted "macro-action" (e.g., "move" or "work") is handed off to a low-level controller trained via Direct Preference Optimization (DPO) to execute the action within a specific simulator.
Datasets
The model was evaluated on two distinct platforms to test its generalization:
- VEET (Virtual Experimental Environment for Trajectories): A Unity-based simulator where human participants navigate rooms and collect coins under varying pressures. This provided real human trajectories and psychometric profiles (e.g., stress, anxiety, depression scales) for 313 participants.
- AgentSociety: A synthetic, LLM-driven city-scale simulation. This served as a "cross-simulator" testbed to see if the same profile-state-behavior factorization could work in a structurally different environment with different action spaces.
Evaluation used a participant-disjoint split, meaning the model was tested on people whose profiles and behaviors were entirely absent from the training set.
Key Insights and Ideas
- Transitions as Supervision: The biggest "aha" moment is that predicting the next state is a powerful signal for predicting the next action. Ablation studies showed that simply adding continuous next-state modeling accounted for the majority of the accuracy gains.
- The Power of Discretization: Using discrete "transition codes" (via Vector Quantization) didn't just help accuracy; it significantly improved "distributional alignment." This means the model didn't just pick the most likely action; it captured the variety and frequency of different human behaviors more accurately.
- Latent Modes $\neq$ Personality Labels: Interestingly, the learned latent codes did not map directly to individual psychometric scales (like anxiety or depression). Instead, they seem to represent complex "behavioral modes"—ways of interacting with the world that are informed by a combination of traits rather than a single label.
- Zero-Shot Generalization: The model successfully transferred its understanding of behavior to "unseen" participants, suggesting it learned generalizable patterns of how human traits translate into environmental interactions.
Key Results
On the VEET dataset (human data):
- Top-1 Accuracy: PsyWorld reached 0.904, a massive jump over the 0.754 achieved by standard Behavioral Cloning (Direct SFT/BC).
- Distributional Similarity: It achieved a near-perfect 0.999 similarity to human action distributions, compared to 0.846 for the baseline.
On the AgentSociety dataset (synthetic data):
- The model maintained its lead, achieving 0.595 Top-1 accuracy vs. 0.525 for the baseline, proving the architecture's robustness across different simulation logic.
Limitations and Future Work
- Not Causal: The authors emphasize that the model identifies predictive associations, not causal psychological mechanisms. It doesn't "explain" why a person acts; it just predicts it better.
- Short-Horizon Focus: While the model is great at next-step prediction, it does not yet establish "calibrated long-horizon state dynamics"—meaning its accuracy might degrade over very long simulated timeframes.
- Affective States: The current model lacks a dynamic "affect" channel (e.g., changing mood during a task), relying instead on stable traits. Integrating time-varying emotions is a suggested next step.
- Data Privacy: The work relies on sensitive psychometric data, necessitating strict de-identification and restricted access protocols.
Summary updated