Factoring Controllable and Exogenous Dynamics in World Models

91 concepts74 papers1 proposal

Imagine an autonomous vehicle trying to navigate a busy city street. To plan its next move, its internal "world model" must predict how the visual scene will change over the next few seconds. A standard deep learning model will attempt to compress the entire high-dimensional image into a single latent vector.

However, the visual scene contains a chaotic mix of information: the car's own steering adjustments, a pedestrian stepping onto the crosswalk, clouds drifting across the sky, and a television screen flickering in a shop window. If the model treats all these pixels equally, it falls into the classic "noisy TV" problem. Because the flickering television or drifting clouds contain high-entropy, unpredictable transitions, the model will waste its limited representational capacity trying to master this background noise, completely losing track of the variables it actually controls.

To solve this, a major research direction focuses on latent state decomposition—explicitly partitioning the world model's latent space into distinct components based on controllability and task relevance. By separating what the agent can influence (endogenous dynamics) from what evolves independently (exogenous dynamics), agents can ignore environmental noise and plan with unprecedented sample efficiency.


The Anatomy of State Decomposition

Instead of mapping an observation into a single, monolithic vector, a factored world model splits the latent space into independent streams. At its core, this architecture separates the state $s$ into an endogenous component $s_{\text{endo}}$ and an exogenous component $s_{\text{exo}}$.

The intuition behind this split lies in how these states evolve over time:

  • Endogenous Transitions: The next endogenous state depends on both the current endogenous state and the agent's action:
    $$s_{\text{endo}, t+1} = f(s_{\text{endo}, t}, a_t)$$
  • Exogenous Transitions: The next exogenous state evolves completely independently of the agent's actions, driven purely by environmental physics or external forces:
    $$s_{\text{exo}, t+1} = g(s_{\text{exo}, t})$$

Vincent Liu et al. formalize this property as Action Impact Regularity (AIR). They demonstrate that in most real-world scenarios, an agent's actions have a localized impact. By treating the exogenous sequence as a fixed, uncontrollable background, the agent can simulate different decision paths entirely within the endogenous subspace without triggering impossible or discontinuous jumps in the environment state.

   ┌─────────────────────────┐
   │    Visual Observation   │
   └────────────┬────────────┘
                │
        ┌───────┴───────┐
        ▼               ▼
 ┌─────────────┐ ┌─────────────┐
 │ Endogenous  │ │ Exogenous   │
 │   (Agent)   │ │ (Background)│
 └──────┬──────┘ └──────┬──────┘
        │               │
   + Action ($a$)   No Action
        │               │
        ▼               ▼
 ┌─────────────┐ ┌─────────────┐
 │ Next State  │ │ Next State  │
 └─────────────┘ └─────────────┘

To make this partitioning even more precise, Yu-Ren Liu et al. introduce the IFactor framework, which divides the world into four distinct causal blocks:

  1. Controllable reward-relevant factors (e.g., the agent's position relative to a goal).
  2. Uncontrollable reward-relevant factors (e.g., an opponent moving to block the goal).
  3. Controllable reward-irrelevant factors (e.g., cosmetic changes the agent can cause but don't affect the score).
  4. Pure noise (e.g., ambient background distractors).

By isolating these blocks, the agent can discard the irrelevant components and focus its planning budget exclusively on what matters.


Disentangling Self from World

A powerful consequence of this factorization is agent-awareness—the ability of an AI to distinguish its own physical "body" from the surrounding environment.

Standard reinforcement learning agents often struggle to understand which pixels belong to them. Kevin Gmelin et al. address this by introducing SEAR (Structured Environment-Agent Representations). By using a structured bottleneck that forces a specific portion of the latent space to reconstruct the agent's physical mask, the remaining latent capacity is freed to model external object dynamics.

Similarly, Manuel Serra Nunes et al. propose Ego-Foresight, a self-supervised auxiliary task where the agent explicitly predicts the visual consequences of its own actions. This allows the model to cleanly decouple self-motion from static background elements, which dramatically improves sample efficiency when the agent must manipulate external tools.


How to Learn the Split Without Labels

In a pure pixel-based environment, the agent isn't given labels showing what is controllable and what is exogenous. The core technical challenge is discovering this split purely from raw trajectories. Researchers have developed three primary mechanisms to achieve this:

1. Multistep Inverse Dynamics

The most prominent mathematical tool for filtering out exogenous noise is the inverse dynamics model. An inverse model tries to predict the action $a_t$ that occurred between two consecutive states: $P(a_t | s_t, s_{t+1})$.

As proven by Yonathan Efroni et al. in their Exogenous Block MDP framework, if a feature in the environment cannot help predict the agent's action across multiple steps, it is mathematically guaranteed to be exogenous. By optimizing the representation to maximize action predictability, the exogenous distractors are naturally filtered out.

2. Counterfactual Blame Assignment

Another intuition relies on variance explanation. Raul Vicente et al. introduce the Controlled Effect Network (CEN), which utilizes a two-stage training procedure. First, an environment-only model is trained to explain as much variance in state transitions as possible without looking at the agent's actions. Then, an action-dependent model is tasked with explaining only the residual variance. This prevents the agent from taking "credit" for independent environmental changes, ensuring its internal model of control is grounded in true counterfactual causality.

3. Exploiting Deterministic Loops

Can an agent learn this split from a single continuous trajectory without environment resets? Alexander Levine et al. demonstrate that it can by exploiting the contrast between deterministic controllable transitions and stochastic exogenous noise. By executing actions that form predictable, closed loops in the environment, the agent can "wait out" the stochastic background noise, separating persistent controllable structures from transient environmental fluctuations.


Why Factorization Works: Practical Benefits

By breaking the monolithic state assumption, factored world models solve several fundamental failure modes in sequential decision-making:

  • Immunity to the Noisy TV: In curiosity-driven exploration, agents are traditionally attracted to unpredictable states. Kaichen Huang et al. show that by isolating endogenous components, curiosity can be directed only toward states the agent can actually influence, preventing the agent from getting paralyzed by background noise.
  • Capacity Preservation: Xiang Fu et al. and Simon Shaolei Du et al. demonstrate that allocating separate latent pathways for task-relevant signals and task-irrelevant distractors prevents complex backgrounds from overwhelming the model's capacity, allowing compact networks to solve highly cluttered visual tasks.
  • Robust Offline Learning: When learning from static, pre-recorded visual datasets, moving backgrounds often cause offline algorithms to become overly conservative. Frameworks like SeMOPO (Shenghua Wan et al.) and SeMAIL (Shenghua Wan et al.) decouple action-dependent and action-independent spaces, allowing agents to accurately imagine alternate decision paths without being misled by irrelevant visual noise.

Ultimately, factoring controllable and exogenous dynamics mimics a fundamental human cognitive trait: the ability to ignore the rustling of leaves in the wind to focus entirely on the ball coming toward us.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_latent_state_disentanglement_bfc6a156,
  title = {latent state disentanglement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/latent-state-disentanglement/rdr_bfc6a1567692462289cdcd0553288505},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_latent_state_disentanglement_bfc6a156_v1,
  title = {latent state disentanglement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/latent-state-disentanglement/rdr_bfc6a1567692462289cdcd0553288505/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake