Planning and Control within Learned Latent Manifolds

17 concepts11 papers1 proposal

When an autonomous agent plans its future actions, it relies on an internal model of the world to predict what will happen next. In an ideal world, this internal simulator would be perfect. In reality, dynamics models and reward functions are approximations learned from data.

This approximation creates a fundamental vulnerability when paired with trajectory optimization. If you task a powerful optimizer with finding a sequence of raw actions or pixel states that maximizes a reward, it acts as an adversary to the internal model. It will inevitably discover the model's blind spots—unrealistic shortcuts or "hallucinations" where the model incorrectly predicts a massive reward or a successful outcome. The planner exploits these inaccuracies, resulting in trajectories that look highly optimized in imagination but fail catastrophically in the real world.

To prevent this failure mode, a growing body of research focuses on manifold-constrained planning. Instead of searching through high-dimensional, unconstrained observation or action spaces, these frameworks force the optimization process to operate entirely within a learned, low-dimensional latent manifold. By restricting the search space to the underlying distribution of valid states and behaviors, the planner is physically regularized, preventing it from exploiting model inaccuracies.

The Vulnerability of Unconstrained Planning

To understand why planning within a latent manifold works, we must first look at why planning in raw spaces fails. Consider an agent trying to plan a long sequence of actions directly from image inputs. If the agent optimizes a sequence of raw actions using an unconstrained dynamics model, tiny errors in predicting the immediate next state accumulate over time.

As Nan Rosemary Ke et al. demonstrate in Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future and Modeling the Long Term Future in Model-Based Reinforcement Learning, standard model-based reinforcement learning suffers heavily from these compounding errors. The optimizer treats the dynamics model's inaccuracies as high-reward shortcuts, steering the agent into unfamiliar states where the model's predictions break down completely.

A similar issue occurs in action space. In Prediction and Control with Temporal Segment Models, Nikhil Mishra et al. note that unconstrained optimization over raw control inputs allows the planner to exploit model inaccuracies. The optimizer selects erratic, unnatural control sequences that the dynamics model cannot reliably predict, leading to real-world failure.

The Latent Manifold as a Natural Regularizer

The core insight of manifold-constrained planning is to use a generative model—such as a Variational Autoencoder (VAE)—to map the complex, high-dimensional space of valid environment states or actions into a compressed, continuous latent space. This latent space acts as a coordinate system for the manifold of reality.

Raw Space (Pixels / High-Dim Torques) 
  [ Infinite configurations, mostly invalid or physically impossible ]
                │
                ▼ (Generative Encoder)
Learned Latent Manifold 
  [ Compressed subspace where every point maps to a valid configuration ]
                │
                ▼ (Optimization Search)
  [ Planner safely navigates without encountering model hallucinations ]

When an optimizer searches for a plan, it does not propose raw actions or raw pixel states. Instead, it searches through this latent space. Because the generative model was trained only on valid, physically feasible data, the decoder guarantees that any point chosen by the optimizer maps back to a realistic state or action.

This mathematical constraint serves as a powerful regularizer. For instance, He Zhang et al. show in GoldenStart: Q-Guided Priors and Entropy Control for Distilling Flow Policies that restricting a policy's input to a low-dimensional latent manifold acts as a shield against critic overestimation. By keeping action generation within a safe subspace identified during training, the agent cannot exploit inaccuracies in the reward function or value estimator.

However, simply mapping to a latent space is not always enough; the optimizer can still drift toward the boundaries of the learned distribution. To counteract this, Chia-Man Hung et al. introduce a path-planning framework called LSPP in Reaching Through Latent Space: From Joint Statistics to Path Planning in Manipulation. They integrate a prior loss directly into the optimization objective, penalizing the planner if it leaves the high-density regions of the training distribution. This explicit constraint ensures that generated robot trajectories remain physically feasible and free of self-collisions without requiring explicit inverse kinematics.

Grounding Subgoals and Kinematics

Manifold constraints are equally effective when planning intermediate milestones, or subgoals, across long time horizons. If a robot tries to plan a sequence of visual subgoals in raw pixel space, the optimization problem is plagued by high dimensionality and noise. More importantly, the planner might generate non-physical intermediate images—like an object teleporting across the screen.

To solve this, Soroush Nasiriany et al. developed LEAP (Planning with Goal-Conditioned Policies), which generates sequences of subgoals within a learned latent embedding space. By planning on the latent manifold, the framework ensures that every generated subgoal corresponds to a physically achievable state.

Similarly, Chelsea Finn et al. present Hierarchical Visual Foresight (HVF) in Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation, optimizing intermediate visual goals within a low-dimensional latent space to bypass the noise of raw pixel optimization. In more abstract reasoning tasks, Doojin Baek et al. leverage a similar principle in Learning to Theorize the World from Observation. Their framework uses "state grounding" to ensure that every intermediate step generated during the execution of a latent program remains strictly within the manifold of valid observations, preventing the model from taking non-physical latent shortcuts.

This concept extends directly to physical hardware and morphology. In First Steps: Latent-Space Control with Semantic Constraints for Quadruped Locomotion, the authors demonstrate that optimizing within a low-dimensional latent space allows a quadruped robot's planned trajectories to automatically inherit the natural physical correlations between its joints. Rather than calculating complex analytical constraints for leg coordination, the planner naturally adheres to them because the latent space itself reflects those physical realities.

At a deeper structural level, Sehoon Ha et al. prove in Joint Optimization of Robot Design and Motion Parameters using the Implicit Function Theorem that representing valid robot configurations as a mathematical constraint manifold allows for the simultaneous optimization of both a robot's physical dimensions (morphology) and its control parameters. Navigating this high-dimensional surface using gradient-based methods ensures that every iteration remains mechanically feasible and compliant with the laws of physics.

Temporal Abstraction and Skill Spaces

Beyond constraining static states or immediate actions, the latent manifold concept can be extended temporally to represent entire behaviors.

Instead of forcing a planner to select individual actions at every single time step, the system can learn a manifold of skills or temporal segments. Nikhil Mishra et al. implement this by optimizing control sequences within a learned latent action manifold composed of extended temporal segments. This drastically reduces the search space and ensures that the planned trajectories match natural behavior patterns seen during training.

When an agent must operate continuously in the real world without external resets, unconstrained exploration can easily lead to "sink states"—such as a robot falling over or getting permanently wedged in a corner. Kevin Lu et al. address this in Reset-Free Lifelong Learning with Skill-Space Planning via their Lifelong Skill Planning (LiSP) framework.

By planning within an abstract, learned skill space rather than a raw action space, LiSP constrains the search to behaviors the agent can reliably execute. This temporal manifold constraint prevents the planner from selecting high-risk or physically impossible actions, significantly reducing the accumulation of model errors over long horizons and enabling safe, lifelong learning.

Key Foundational Work

Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future (2019) — contributed 2 concept elements to this direction.
Planning with Goal-Conditioned Policies (2019) — contributed 2 concept elements to this direction.
Modeling the Long Term Future in Model-Based Reinforcement Learning (2019) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_manifold_constrained_planning_fa28b0df,
  title = {manifold-constrained planning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/manifold-constrained-planning/rdr_fa28b0df2f6f4265a71dc0d3800525b7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_manifold_constrained_planning_fa28b0df_v1,
  title = {manifold-constrained planning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/manifold-constrained-planning/rdr_fa28b0df2f6f4265a71dc0d3800525b7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake