Model Predictive Control via Stochastic Trajectory Sampling

31 concepts30 papers1 proposal

When controlling a complex system—whether it is a rally car drifting around a loose gravel turn, a robot folding a piece of fabric, or an automated ventilator stabilizing a patient's breathing—we face a fundamental challenge: how do we find the optimal sequence of actions when the underlying physics are highly non-linear, discontinuous, or learned entirely from data?

Traditional optimal control relies on analytical models and gradient-based optimization. However, when our understanding of the world is captured by deep neural networks, computing exact gradients through time becomes notoriously difficult. Deep networks introduce highly non-linear, non-convex landscapes where gradient descent easily gets trapped in local minima or fails due to exploding and vanishing gradients.

Sampling-based Model Predictive Control (MPC) bypasses this mathematical bottleneck by replacing analytical optimization with parallelized simulation. Instead of calculating how to change an action sequence via gradients, the system stochastically generates thousands of candidate action sequences, rolls them forward through a predictive model to "imagine" their outcomes, scores them against a cost function, and selects the best path.

By treating the predictive model as a black box, this framework decouples the complexity of the environment's dynamics from the optimization process itself.


The Core Architecture: Sample, Simulate, Score

The mechanics of sampling-based MPC operate as a continuous loop driven by three core steps:

  [ Current State ]
          │
          ▼
┌────────────────────────┐
│ 1. Sample Trajectories │ ◄─── Update distribution (e.g., CEM)
└─────────┬──────────────┘
          │ (Thousands of action sequences)
          ▼
┌────────────────────────┐
│ 2. Simulate Futures    │ ───► Using a Forward World Model
└─────────┬──────────────┘
          │ (Predicted future states)
          ▼
┌────────────────────────┐
│ 3. Score & Select      │ ───► Evaluate against Cost/Reward Map
└─────────┬──────────────┘
          │ (Execute first action only)
          ▼
  [ Environment Step ]
  1. Sample: The controller generates a large batch of candidate action sequences over a finite future horizon. These sequences are drawn from a proposal distribution (initially random or informed by the previous time step's plan).
  2. Simulate (Forward Rollout): Each candidate action sequence is fed into a forward dynamics model. The model predicts the sequential evolution of states over the planning horizon, effectively generating thousands of imagined futures in parallel.
  3. Score and Select: The imagined trajectories are evaluated against a multi-objective cost surface or reward model. The controller then selects the optimal sequence—or updates its sampling distribution to favor higher-scoring regions—and executes the very first action of that sequence.

This process is repeated at every single time step. As highlighted by Drews et al., evaluating thousands of potential future trajectories against a cost surface in real-time allows autonomous vehicles to navigate highly non-linear driving regimes at the absolute limits of handling. Similarly, Liu et al. demonstrate that model-based agents can completely bypass task-specific policies by using MPC to plan actions directly through an online world model.


The Forward Model: Imagining the Future

The accuracy of a sampling-based planner is fundamentally bounded by its forward model. Because the planner relies entirely on virtual rollouts to evaluate actions, the model must accurately capture how the environment responds to control inputs. Researchers have developed several distinct strategies for structuring these forward models depending on the state space:

  • Visual Foresight: For tasks where states are represented as raw pixels, the forward model acts as a video predictor. Finn et al. pioneered this by combining deep action-conditioned video prediction with MPC, allowing a robot to stochastically sample action sequences and evaluate outcomes based on predicted pixel movements.
  • Object-Centric Graph Networks: Rather than predicting every pixel in a scene, forward models can represent the world as a collection of discrete entities. Ye introduced an object-centric forward model that uses graph-based interaction networks to predict how entities collide and move, enabling highly efficient multi-step manipulation planning.
  • Structured Physical Mechanics: When dealing with complex materials like sand or beans, unstructured networks often struggle. Schenck et al. showed that convolutional networks explicitly structured to model the physical mechanics of scooping and dumping significantly outperform standard value-based models.
  • Recurrent Neural Networks: To capture temporal dependencies and human-like adaptation, forward models can be built using recurrent architectures. Weinstein et al. utilized an RNN as an adaptive forward model within a model-based architecture to successfully replicate human motor control and structure-learning behavior.

Optimization Without Gradients

Once a forward model is established, how do we efficiently search the space of possible action sequences without relying on gradients? The literature highlights three primary zero-order optimization strategies:

Random Shooting

The simplest approach is random shooting, where action sequences are sampled uniformly at random, rolled forward, and the single best sequence is chosen. While computationally trivial, it scales poorly as the planning horizon or action dimensionality increases.

The Cross-Entropy Method (CEM)

To focus computational power on promising trajectories, CEM refines the sampling distribution iteratively. It samples a batch of trajectories, selects the top-performing fraction (the "elite" set), and updates the mean and variance of the sampling distribution to match these elites.

This iterative refinement is used extensively across visual planning frameworks (Finn et al.), continual world models (Liu et al.), and brain-inspired active inference architectures (Tschantz et al.).

Path Integral (PI) Control

Rather than discarding non-elite trajectories, Path Integral methods compute a weighted average of all sampled trajectories, where each path's weight is exponentially proportional to its performance. This creates a smoother optimization landscape.

However, generating enough samples to accurately approximate this optimal control law can be demanding. To address this, Yoon et al. derived explicit sampling complexity bounds for PI methods, establishing a theoretical foundation for how system dynamics and cost functions dictate the minimum number of random samples required to guarantee performance.

As noted by Vlastelica et al., these zero-order approaches provide immense robustness when planning through the complex, non-linear dynamics represented by deep neural network ensembles, as they are entirely unaffected by discontinuous or rugged cost landscapes.


A major advantage of sampling-based MPC is its capacity to handle risk, safety constraints, and environmental uncertainty explicitly during the planning phase. Because the controller evaluates full trajectories before executing them, it can proactively steer the system away from dangerous states.

Parameter and Model Uncertainty

Learned models are inevitably imperfect. To bridge the "reality gap" between simulation and the physical world, Abraham et al. utilized an ensemble of physics simulators—each initialized with different physical parameters like mass or friction—to evaluate action sequences. By aggregating costs across this diverse ensemble, the controller identifies trajectories that are robust to environmental variations.

Furthermore, Wu et al. developed the "Plan to Predict" framework, which trains world models to foresee their own long-horizon uncertainty, allowing the MPC planner to actively avoid regions of high epistemic uncertainty.

Active Probing and Information Gain

Uncertainty can also be reduced through action. Knaup et al. developed a Dual Model Predictive Path-Integral (DMPPI) controller that includes an information-gain metric directly within the trajectory cost function. This allows an autonomous vehicle to execute strategic, subtle maneuvers that actively "probe" surrounding human drivers to deduce their driving style, significantly improving safety during complex highway merging scenarios.

Hazard Avoidance and Risk Functions

In safety-critical domains, the cost function can be augmented with specialized risk metrics or human feedback. For instance, Rahtz et al. utilized MPC to evaluate potential action sequences within a learned simulator to maximize predicted rewards while strictly avoiding hazards identified by human supervisors.

In the medical domain, O'Kane et al. applied a similar look-ahead planning strategy to blood glucose control for Type 1 Diabetes, evaluating candidate insulin dosing sequences against a specialized risk function to proactively prevent dangerous hypoglycemic states.


Closing the Loop: Receding Horizon Control

Why does sampling thousands of future steps work in volatile, real-world environments? The secret lies in Receding Horizon Control.

If a robot were to execute an entire sampled trajectory open-loop, compounding errors in the learned dynamics model would quickly cause the system to drift off course. By executing only the immediate next action and repeating the entire sampling and optimization process at the next time step, the system continuously corrects for model inaccuracies and unexpected environmental shifts.

This reactive, high-frequency replanning loop enables success across an incredibly diverse array of complex applications:

  • Underactuated and Nonholonomic Systems: Lambert et al. demonstrated that combining a learned dynamics model with sample-based trajectory optimization allows a 30-milligram flying micro-robot to discover complex, multi-step oscillatory maneuvers required to actuate degrees of freedom that lack direct control (such as yaw).
  • Deformable Object Manipulation: Hoque et al. leveraged online optimization over learned dynamics to enable robots to sequentially smooth and fold highly deformable fabrics without requiring explicit 3D mesh tracking.
  • Real-Time Edge Adaptation: Tsuchiya combined sampling-based control with meta-learning, allowing autonomous vehicles to adapt their internal dynamics models to changing road conditions on the fly, ensuring precise control at the limits of friction.
  • High-Dimensional Motion Planning: Rather than following noisy gradients, Ni et al. used sampling-based MPC to select optimal next states based on a learned cost-to-go field, allowing high-degree-of-freedom robots to fluidly navigate cluttered environments.

From managing building HVAC schedules (Zhang et al.) and ensuring safe autonomous drone flight (Kang et al.) to optimizing off-road driving smoothness (Manderson et al.), automating mechanical ventilators (Ward et al.), executing fluid human-to-robot handovers (the authors), and learning from state-space human feedback (the authors), the underlying intuition remains identical. By pairing the expressive power of deep learning forward models with the brute-force parallelized imagination of stochastic sampling, modern controllers achieve robust, highly adaptive, and optimal behavior across the physical world.

Key Foundational Work

Aggressive Deep Driving: Combining Convolutional Neural Networks and Model Predictive Control (2017) — contributed 1 concept elements to this direction.
Structure Learning in Motor Control: A Deep Reinforcement Learning Model (2017) — contributed 1 concept elements to this direction.
Deep Visual Foresight for Planning Robot Motion (2016) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_sampling_based_model_predictive_control_ef04775a,
  title = {sampling-based model predictive control},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sampling-based-model-predictive-control/rdr_ef04775af7d048a9af4eda77d679f7ed},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_sampling_based_model_predictive_control_ef04775a_v1,
  title = {sampling-based model predictive control},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sampling-based-model-predictive-control/rdr_ef04775af7d048a9af4eda77d679f7ed/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake