Adaptive Linear Path Model-Based Diffusion

Yutaka Shimizu, Masayoshi Tomizuka

TL;DR

This paper introduces LP-MBD, a simplified diffusion-based control framework that uses a linear probability path to decouple noise scale from sampling steps. Building on this, the authors present ALP-MBD, which uses reinforcement learning to dynamically adjust these parameters on-the-fly, allowing robots to use more computational effort for complex maneuvers and less for simple ones.

What Problem Does This Paper Address?

Diffusion models have recently become a "hot" tool for robotic trajectory optimization because they excel at handling non-convex constraints and multi-modal distributions (e.g., deciding whether to pass an obstacle on the left or right). However, they have a major Achilles' heel: parameter sensitivity.

Standard diffusion uses a "variance-preserving" (VP) noise schedule. In this setup, the number of diffusion steps ($T$) and the noise levels ($\beta_0, \beta_1$) are intricately coupled. If you want to increase the resolution of your trajectory by increasing $T$, you inadvertently change the entire noise profile. This makes manual tuning a nightmare. Furthermore, most systems use a fixed schedule. This is inefficient; a robot driving on a straight, empty road doesn't need the same "computational depth" (diffusion steps) as a robot navigating a tight, obstacle-ridden alleyway.

Methods

The authors propose two main innovations:

  1. Linear Path Model-Based Diffusion (LP-MBD): Inspired by "Flow Matching," they replace the complex VP schedule with a simple linear interpolation between a clean trajectory and Gaussian noise.
  2. The "Aha!" Moment: This formulation decouples the maximum noise scale ($\sigma_{max}$) from the number of steps ($T$). You can now change the "resolution" of your search without changing the "range" of your exploration.
  3. Adaptive LP-MBD (ALP-MBD): Because LP-MBD provides a stable, decoupled parameter space, the authors could finally apply Reinforcement Learning (RL) to the scheduler itself.
  4. They trained a Proximal Policy Optimization (PPO) agent to act as a "meta-controller." At each time step, this agent looks at the robot's state and outputs the optimal $T$ and $\sigma_{max}$ for the next diffusion process.
  5. To ensure efficiency, they added a penalty to the RL reward function for using too many diffusion steps.

Datasets

The researchers evaluated their approach across three distinct domains:

  • Numerical Studies: 1D and 2D toy problems involving Gaussian mixtures to visualize how the model handles multi-modal objectives.
  • Brax Benchmarks: A suite of physics-based tasks including Ant, Hopper, HalfCheetah, Walker2D, Reacher, and Pusher.
  • Mobile Robot Trajectory Tracking: A custom environment where a kinematic vehicle must follow S-shaped and oval paths while avoiding static obstacles.

Key Insights and Ideas

  • Decoupling is the Key to Adaptation: The authors argue that applying RL to standard VP-MBD is unstable because small parameter updates cause unpredictable shifts in the noise profile. By switching to a linear path, the parameter space becomes "well-behaved" enough for an RL agent to learn effectively.
  • Geometric Interpretability: The linear path provides a clear, straight-line interpolation between distributions. This makes the diffusion process more like "optimal transport," which is theoretically more grounded than the "curved" paths of standard variance-preserving schedules.
  • Computational Economy: ALP-MBD demonstrates that robots can be "lazy" in a smart way. By reducing $T$ in simple scenarios, the system saves significant computation without sacrificing the quality of the final control signal.

Key Results

  • Performance: In Brax benchmarks, LP-MBD achieved competitive or superior rewards compared to standard VP-MBD and traditional methods like MPPI. For example, in the HalfCheetah task, LP-MBD scored 2.73 vs. VP-MBD's 2.53.
  • Adaptability: In the mobile robot tests, ALP-MBD successfully learned to increase both $T$ and $\sigma_{max}$ when approaching obstacles or entering sharp turns, and decrease them during steady cruising.
  • Efficiency: ALP-MBD achieved the highest average single-episode reward (-3342) compared to fixed-parameter baselines, while maintaining a real-time-capable latency of roughly 45.5 ms per control step.
  • Generalization: An ALP-MBD agent trained on an S-curve path successfully generalized to an unseen oval path, correctly identifying that it needed more diffusion steps to handle the tighter corners.

Limitations and Future Work

The authors noted one specific outlier: the Pusher environment. In this high-dimensional manipulation task, the standard VP-MBD actually outperformed the new LP-MBD. They hypothesize that for very high-dimensional spaces, the "expressive capacity" of the variance-preserving schedule might be superior to a simple linear path.

Future work will likely involve investigating this discrepancy and exploring more complex, non-linear probability paths that might combine the best of both worlds: the interpretability of flow matching with the expressive power needed for complex manipulation.

Summary updated

Abstract

The interest in combining model-based control approaches with diffusion models has been growing. Although we have seen many impressive robotic control results in difficult tasks, the performance of diffusion models is highly sensitive to the choice of scheduling parameters, making parameter tuning one of the most critical challenges. We introduce Linear Path Model-Based Diffusion (LP-MBD), which replaces the variance-preserving schedule with a flow-matching-inspired linear probability path. This yields a geometrically interpretable and decoupled parameterization that reduces tuning complexity and provides a stable foundation for adaptation. Building on this, we propose Adaptive LP-MBD (ALP-MBD), which leverages reinforcement learning to adjust diffusion steps and noise levels according to task complexity and environmental conditions. Across numerical studies, Brax benchmarks, and mobile-robot trajectory tracking, LP-MBD simplifies scheduling while maintaining strong performance, and ALP-MBD further improves robustness, adaptability, and real-time efficiency.
BibTeX
@misc{shimizu2026adaptive,
  author = {Yutaka Shimizu and Masayoshi Tomizuka},
  title = {{Adaptive Linear Path Model-Based Diffusion}},
  year = {2026},
  eprint = {2602.02831},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.02831},
}
Figure