Abstract
In this paper we develop a novel, discrete-time optimal control framework for mechanical systems with uncertain model parameters. We consider finite-horizon problems where the performance index depends on the statistical moments of the stochastic system. Our approach constitutes an extension of the original Differential Dynamic Programming method and handles uncertainty through generalized Polynomial Chaos (gPC) theory. The developed iterative scheme is capable of controlling the probabilistic evolution of the dynamic system and can be used in planning and control. Moreover, its scalable and fast converging nature play a key role when dealing with complex, high-dimensional problems. Based on Lagrangian mechanics principles, we also prove that Variational Integrators can be designed to properly propagate and linearize the gPC representations of stochastic, forced mechanical systems in discrete time. We utilize this benefit to further improve the efficiency of our trajectory-optimization methodology. Numerical simulations are included to validate the applicability of our approach.
Introduction
In classical robotics, trajectory planning often assumes a "ground truth" model where physical parameters—such as the mass of a quadrotor or the friction coefficient of a joint—are known exactly. In reality, these parameters are frequently subject to parametric uncertainty. While standard stochastic control methods like iLQG or SDDP effectively handle external additive noise (e.g., wind gusts), they struggle with uncertainties embedded directly within the system's governing physics equations.
To address this, we introduce gPC-DDP, a framework that transforms complex stochastic trajectory optimization into a computationally efficient deterministic problem. By integrating generalized Polynomial Chaos (gPC) with Differential Dynamic Programming (DDP), the system can explicitly minimize both path error and state variance, ensuring robust performance under model uncertainty.
Methodology: The gPC-DDP Framework
The gPC-DDP approach is built upon three core mathematical pillars designed to handle uncertainty while maintaining physical consistency.
1. Generalized Polynomial Chaos (gPC)
Instead of relying on computationally expensive Monte Carlo simulations to sample possible outcomes, gPC represents the stochastic state of a system as an expansion of orthogonal polynomials. This technique converts a single stochastic differential equation into a set of coupled deterministic equations. The "expanded state" now includes the mean, variance, and higher-order moments of the system's probability distribution.
As shown in Table 1, different types of parametric distributions (e.g., Gaussian, Uniform, Gamma, Beta) are mapped to specific polynomial bases (e.g., Hermite, Legendre, Laguerre, Jacobi). This allows for a tailored representation of the underlying uncertainty within its specific domain.
2. Variational Integrators for Stochastic Systems
For mechanical systems, standard numerical integration methods (like Euler or Runge-Kutta) can suffer from energy drift or instability over long horizons. A key finding of this work is that the coefficients of a gPC expansion for a mechanical system themselves behave like a Lagrangian system. By utilizing Variational Integrators (VI), the framework preserves the geometric and physical properties of the stochastic system, leading to superior numerical stability and accuracy even with large time steps.
3. The gPC-DDP Algorithm
The framework extends the classic Differential Dynamic Programming (DDP) algorithm to the high-dimensional expanded state space. This allows for locally quadratic convergence to the optimal trajectory.
Algorithm 1 outlines the propagation of the Discrete Euler-Lagrange (DEL) equations, enabling the efficient computation of optimal control inputs that account for the probabilistic evolution of the system. This iterative scheme is capable of handling the high-dimensional "expanded" state space created by the gPC expansion.
Numerical Validation and Results
The framework was validated on two simulated benchmarks to demonstrate its efficiency and robustness:
- The Duffing Oscillator: A classic nonlinear system used to demonstrate how the algorithm handles complex, non-Gaussian probability distributions. The algorithm achieved locally quadratic convergence, reaching the optimal solution in roughly 12 iterations.
- 12-DOF Quadrotor: A high-dimensional system where motor constants ($G_{tr}, G_{rot}$) were treated as uncertain and uniformly distributed. gPC-DDP successfully guided the vehicle to a target while keeping the "cloud" of possible trajectories tightly clustered, significantly reducing the standard deviation of the final state compared to standard DDP.
Furthermore, the Variational Integrator version (VI-gPC-DDP) was shown to be much more insensitive to the choice of time step ($\Delta t$) compared to standard Euler-based methods, allowing for successful planning where simpler methods failed.
Conclusion and Future Work
gPC-DDP offers a powerful model-based alternative for stochastic systems, providing fast convergence and physical accuracy. By tracking statistical moments explicitly, the robot can find the "most certain" path rather than just the fastest one.
However, challenges remain regarding the "curse of dimensionality" as the number of gPC coefficients grows exponentially with the number of uncertain parameters. Future work will explore sparse quadrature methods to handle higher-dimensional parameter spaces and multi-element gPC methods to maintain accuracy over very long time horizons where probability distributions become increasingly complex.
Summary updated