The Failure of Dynamic State Tracking in Language Models

45 concepts38 papers1 proposal

Large language models (LLMs) are remarkably fluent, yet they frequently struggle with tasks that require maintaining an accurate "mental map" of a changing environment. While a model might correctly describe a starting state and a final goal, it often loses track of the intermediate transformations—a phenomenon known as state tracking failure. This disconnect between linguistic fluency and logical consistency suggests that current architectures lack a robust internal world model.

The Gap Between Calculation and Understanding

The core of the problem lies in the difference between sequence completion and causal simulation. As Valmeekam et al. demonstrate in the PlanBench benchmark, even advanced models like GPT-4 struggle with basic planning when they must reason about change. A model might propose an action that is physically impossible—such as picking up an object that is already inside a closed container—because it fails to track the logical consequences of its previous steps.

This failure is not limited to physical robotics. In mathematical reasoning, Opedal et al. found that models could often calculate a correct final numerical answer to a story problem while simultaneously failing to answer simple questions about the intermediate states of the entities involved. This suggests that the models are "calculating" without truly "understanding" the underlying world state described in the text.

The Mechanics of State Drift

Why do these models lose their way? In multi-turn interactions, models often suffer from "state drift" or "belief traps." Zou et al. describe this as a situation where an agent's internal model of the environment deviates from reality over time, causing subsequent actions to be based on a flawed premise.

This drift is particularly evident in structured environments:

  • Games: Feng et al. show that models playing chess frequently generate illegal moves because they cannot maintain a stable internal representation of the board state.
  • Interactive Narratives: In the NCP-Bench framework, Ma et al. found that models often collapse into logical contradictions—such as characters or objects changing properties without explanation—when a player subverts the expected plot.
  • Visual Reasoning: Even Vision-Language Models (VLMs) suffer from a form of "tunnel vision." Berman et al. observed that models fail at multi-step visual tasks, such as tracing a wire, because they cannot maintain an internal state that connects information across different parts of an image.

The Memory Wall and Precondition Neglect

A significant bottleneck in state tracking is what the authors of the Berkeley Function Calling Leaderboard call the "memory wall" Patil et al.. In complex tool-use tasks, models struggle to retrieve and update information from previous outputs, leading to hallucinations.

This often manifests as a neglect of physical or logical preconditions. Li et al. argue that many planning failures in embodied AI stem from the model's inability to track state transitions, such as forgetting that a door must be opened before a room can be entered. Similarly, Xia et al. show that even if a model can "find" information in its context, it often fails to manipulate or update that information as the situation evolves.

Strategies for Robust State Tracking

To overcome these limitations, researchers are moving toward "model-based" reasoning, where the state is explicitly managed rather than left to the model's latent representations.

  1. Explicit State Probing: Toshniwal et al. introduced "Baked-in State Probing," which forces models to explicitly output the world state as text during training. This significantly improves the logical consistency of generated sequences.
  2. Symbolic World Models: Frameworks like Statler Yoneda et al. and DELTA Liu et al. maintain an external, symbolic representation of the world (such as a 3D scene graph). The LLM acts as a "reader" and "writer" for this state, ensuring the model's plans remain grounded in reality.
  3. Visual Thinking: Borazjanizadeh et al. found that allowing models to generate and "look at" their own conceptual diagrams helps them maintain a stable mental model during long-horizon planning, enabling better error correction.
  4. Latent Prediction: Rather than relying on external symbols, the authors of the NextLat objective propose training Transformers to predict their own future internal representations. This forces the model to compress history into compact "belief states," leading to more consistent world modeling.

Finally, Zhongqi Yue argues for expanding the action space of LLMs beyond language, allowing them to interact with tools that manage state directly. By shifting from pure sequence prediction to active state management, these approaches aim to bridge the gap between talking about the world and understanding how it works.

Key Foundational Work

Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change) (2022) — contributed 3 concept elements to this direction.
Baked-in State Probing (2022) — contributed 1 concept elements to this direction.
Improving Neural Program Synthesis with Inferred Execution Traces (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_state_tracking_failure_2957e1b9,
  title = {state tracking failure},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/state-tracking-failure/rdr_2957e1b91e6f4d16b8e9317e036a7080},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_state_tracking_failure_2957e1b9_v1,
  title = {state tracking failure},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/state-tracking-failure/rdr_2957e1b91e6f4d16b8e9317e036a7080/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake