TL;DR
This paper introduces latent-SR, a world model that predicts how environments change over long sequences of actions by treating the process as "structured reasoning." Instead of looping a one-step model (which compounds errors) or jumping straight to a distant future (which loses detail), it generates a sequence of sparse textual edits in a single pass, achieving state-of-the-art performance on complex text-based benchmarks like ScienceWorld and CEO-Bench.
What Problem Does This Paper Address?
For an AI agent to plan, it needs a "world model"—a way to simulate what will happen if it takes a specific sequence of actions. Doing this over long horizons is notoriously difficult.
Traditionally, researchers use two methods:
- Recursive Rollouts: The model predicts one step, feeds that prediction back into itself, and repeats. The problem? Small errors in the first step snowball into nonsense by step ten (the "compounding error" problem).
- Direct Multi-step Prediction: The model looks at the current state and a long list of actions, then tries to jump straight to the final result. The problem? It’s hard to learn because the model doesn't "see" the intermediate steps, making it difficult to assign credit for what went right or wrong.
The authors aim to find a middle ground: a model that avoids the feedback loops of recursive rollouts but retains the step-by-step clarity needed for accurate long-term tracking.
Methods
The core innovation is Structured Reasoning (SR). The authors treat the evolution of the world state as a single, structured textual trajectory.
- Sparse Edits ($\Delta$): Instead of regenerating the entire world state at every step, the model predicts only what changed (e.g., "the door is now open"). These "deltas" are applied to a persistent state. This reduces the "state tracking" burden on the model.
- Non-Recursive Trajectory: The model generates the entire sequence of edits ($\Delta_1, \dots, \Delta_k$) in one autoregressive decode. Crucially, these predicted states are never fed back into the model as input during the transition, which breaks the cycle of compounding errors.
- Two-Stage Training:
- Distillation: The model is first trained to imitate high-quality trajectories generated by a "teacher" model (GPT-5.2).
- Trajectory-Level RL: The model is refined using Group Relative Policy Optimization (GRPO). It receives rewards not just for the final prediction, but for predictive gain (how much better the state is than just looking at raw history) and intermediate quality (checking if the state is accurate at every step along the way).
Datasets
The authors evaluated their approach on three diverse, partially observable environments:
- SCIENCEWORLD: A complex simulator for elementary science tasks (e.g., boiling water, growing plants).
- JERICHO: A collection of classic interactive fiction games (like Zork).
- CEO-BENCH: A synthetic business simulation where agents manage a company's cash, subscribers, and service health over many weeks.
Key Insights and Ideas
- Text as a "White Box" Latent State: Unlike continuous vector-based world models, textual states are human-readable. This allows the researchers to supervise and reward the model's "internal thoughts" about the world state directly.
- The Power of Sparsity: By only focusing on changes ($\Delta$), the model acts like a version control system (like Git). It doesn't need to remember that the sky is blue if the action was "open the drawer"; it only needs to record the drawer's new state.
- Action Sensitivity: A major finding was that many baseline models are "action-blind"—they predict the same future regardless of what actions are taken. By forcing the model to reason through intermediate edits,
latent-SRbecomes significantly more sensitive to the specific actions provided.
Key Results
latent-SR consistently outperformed both recursive and direct baselines, with the gap widening as the prediction horizon increased.
- ScienceWorld: At a 10-step horizon,
latent-SRimproved Fact-F1 scores by 12.3% relative to the strongest baseline. - CEO-Bench: Achieved a 9.3% relative gain at horizon 10.
- Jericho: Reduced prediction error by nearly 30%.
In a controlled "counterfactual" study, latent-SR was the only model that showed statistically significant sensitivity to future actions, whereas baselines often failed to distinguish between different action sequences.
Limitations and Future Work
- Text-Centric: The current framework relies on the environment being representable via text. Extending this to visual or continuous control domains (like robotics) remains an open challenge.
- Teacher Dependence: The initial distillation phase requires a high-quality teacher model to generate the first "reasoning traces."
- Closed-Loop Planning: While the paper proves
latent-SRis a better model of the world, the authors haven't yet integrated it into a full planning agent to see if it leads to better actions in a closed-loop setting.
Summary updated