Factored State Spaces for Mixed Observability Planning

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 220 papers27 conceptsfrozen 2026-10-04

In the classic formulation of a Partially Observable Markov Decision Process (POMDP), an agent is essentially "blind," navigating a high-dimensional cloud of probability known as a belief space. For a robot with dozens of sensors and a complex environment, this belief space becomes an intractable nightmare—the "curse of dimensionality" makes optimal planning nearly impossible.

However, researchers have observed that robots are rarely completely blind. A self-driving car might not know if a pedestrian will step into the street, but it knows its own velocity and GPS coordinates with high certainty. By exploiting this "mixed observability," we can factor the world into what we know for sure and what we must guess, drastically simplifying the math of autonomy.

The Geometry of Certainty

The core breakthrough, formalized by SM Ong, David Hsu, and colleagues in POMDPs for robotic tasks with mixed observability, is the Mixed Observability MDP (MOMDP).

In a standard POMDP, the belief state is a probability distribution over every possible state of the system. If a system has $n$ states, the belief space is an $(n-1)$-dimensional simplex. In an MOMDP, we partition the state space into two components:

  1. Fully Observable ($x$): Variables like the robot’s joint angles, battery level, or current position.
  2. Partially Observable ($y$): Latent variables like the weight of an object, the intent of a human, or the location of a hidden target.

By factoring the state this way, the robot only needs to maintain a belief distribution over $y$. As David Hsu and Wee Sun Lee demonstrated in What makes some POMDP problems easy to approximate?, this decomposition reduces the "covering number" of the reachable belief space—a measure of how many points are needed to approximate the solution—by a doubly exponential factor. This is why MOMDP solvers can be orders of magnitude faster than general POMDP solvers.

Learning by Doing in Physical Spaces

This factored approach allows robots to interact with the physical world more naturally. In Multi-modal Predicate Identification using Dynamically Learned Robot Controllers, Saeid Amiri and Shiqi Zhang show how a robot can identify object properties like "hollowness" or "heaviness." The robot’s own configuration is $x$ (known), while the object’s property is $y$ (hidden). The MOMDP framework allows the robot to plan a sequence of "pokes" or "shakes" specifically designed to collapse the uncertainty of $y$ without wasting energy re-estimating its own arm position.

Similarly, in multi-agent scenarios like target hunting, the Mixed Observable RRT (MORRT) algorithm developed by Aaron Ames and Ugo Rosolia treats robot positions as deterministic. This allows the planner to focus entirely on the probabilistic "heat map" of the hidden target, generating contingency plans that prioritize information gain.

Beyond Robotics: Adaptive Management

The power of mixed observability extends to "Adaptive Management" (AM)—fields like conservation biology or public health where we must make decisions while learning how the environment works. Iadine Chades and Sam Nicol argue in MOMDPs: A Solution for Modelling Adaptive Management Problems that these problems are naturally MOMDPs.

In ecological management, the current state of a forest might be visible ($x$), but the underlying biological growth model is unknown ($y$). By using MOMDP solvers, managers can "learn by doing," choosing actions that balance immediate conservation goals with the long-term value of discovering which biological model is correct. To make these solutions more practical for human decision-makers, Jonathan Ferrer-Mestres and Tom Dietterich introduced K-N-MOMDPs, which simplify the observable and hidden states independently to create interpretable policies.

Modeling Hidden Human States

Mixed observability is also a key tool for human-AI collaboration. In Shared Autonomy Systems with Stochastic Operator Models, Clarissa Costen and Nick Hawes model a human operator's fatigue or skill level as a hidden variable. The AI observes the environment ($x$) and the human's inputs, using an MOMDP to infer the operator's internal state ($y$). This allows the AI to decide when to take over control or when to provide assistance, based on a real-time estimate of the human's reliability.

This concept of "inferring reliability" is echoed in work by Dylan M. Asmar and Mykel Kochenderfer in Learning to Trust. By separating the query budget (known) from the advisor's reliability (hidden), agents can strategically decide when to "pay" for advice, optimizing their long-term performance in uncertain environments.

The Future: Learning Mixed Models

While early work focused on solving MOMDPs with known models, recent research has moved toward learning these structures from scratch. The Mixed Observability Predictive State Representation (MO-PSR) by Sylvie C. W. Ong and Joelle Pineau provides a way to learn system dynamics that respect this split. By acknowledging that some parts of the observation space are noise-free, MO-PSRs create models that are more compact and accurate than those that treat every variable as hidden.

As summarized in the survey by Mikko Lauri and Joni Pajarinen, Partially Observable Markov Decision Processes in Robotics: A Survey, the transition from general POMDPs to structured, mixed-observable models is what has allowed probabilistic planning to move from simple grid-worlds to complex, real-world robotic systems. By admitting what we know, we gain the computational power to reason about what we don't.

Papers in this version

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 16 papers
  2. Version 1 · 2026-10-04 · 20 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_mixed_observability_modeling_a82a2031,
  title = {mixed observability modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mixed-observability-modeling/rdr_a82a20315f11435d985a24430bfb60f6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_factored_state_spaces_for_mixed_observab_a82a2031_v1,
  title = {Factored State Spaces for Mixed Observability Planning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/factored-state-spaces-for-mixed-observability-planning/rdr_a82a20315f11435d985a24430bfb60f6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}