Stochastic trajectory optimization for mechanical systems with\n parametric uncertainties

George I. Boutselis, Yunpeng Pan, Gerardo De La Tore, Evangelos A. TheodorouarXiv 2017 · DOI · arXiv

Abstract

In this paper we develop a novel, discrete-time optimal control framework for mechanical systems with uncertain model parameters. We consider finite-horizon problems where the performance index depends on the statistical moments of the stochastic system. Our approach constitutes an extension of the original Differential Dynamic Programming method and handles uncertainty through generalized Polynomial Chaos (gPC) theory. The developed iterative scheme is capable of controlling the probabilistic evolution of the dynamic system and can be used in planning and control. Moreover, its scalable and fast converging nature play a key role when dealing with complex, high-dimensional problems. Based on Lagrangian mechanics principles, we also prove that Variational Integrators can be designed to properly propagate and linearize the gPC representations of stochastic, forced mechanical systems in discrete time. We utilize this benefit to further improve the efficiency of our trajectory-optimization methodology. Numerical simulations are included to validate the applicability of our approach.

Introduction

In classical robotics, trajectory planning often assumes a "ground truth" model where physical parameters—such as the mass of a quadrotor or the friction coefficient of a joint—are known exactly. In reality, these parameters are frequently subject to parametric uncertainty. While standard stochastic control methods like iLQG or SDDP effectively handle external additive noise (e.g., wind gusts), they struggle with uncertainties embedded directly within the system's governing physics equations.

To address this, we introduce gPC-DDP, a framework that transforms complex stochastic trajectory optimization into a computationally efficient deterministic problem. By integrating generalized Polynomial Chaos (gPC) with Differential Dynamic Programming (DDP), the system can explicitly minimize both path error and state variance, ensuring robust performance under model uncertainty.

Methodology: The gPC-DDP Framework

The gPC-DDP approach is built upon three core mathematical pillars designed to handle uncertainty while maintaining physical consistency.

1. Generalized Polynomial Chaos (gPC)

Instead of relying on computationally expensive Monte Carlo simulations to sample possible outcomes, gPC represents the stochastic state of a system as an expansion of orthogonal polynomials. This technique converts a single stochastic differential equation into a set of coupled deterministic equations. The "expanded state" now includes the mean, variance, and higher-order moments of the system's probability distribution.

As shown in Table 1, different types of parametric distributions (e.g., Gaussian, Uniform, Gamma, Beta) are mapped to specific polynomial bases (e.g., Hermite, Legendre, Laguerre, Jacobi). This allows for a tailored representation of the underlying uncertainty within its specific domain.

2. Variational Integrators for Stochastic Systems

For mechanical systems, standard numerical integration methods (like Euler or Runge-Kutta) can suffer from energy drift or instability over long horizons. A key finding of this work is that the coefficients of a gPC expansion for a mechanical system themselves behave like a Lagrangian system. By utilizing Variational Integrators (VI), the framework preserves the geometric and physical properties of the stochastic system, leading to superior numerical stability and accuracy even with large time steps.

3. The gPC-DDP Algorithm

The framework extends the classic Differential Dynamic Programming (DDP) algorithm to the high-dimensional expanded state space. This allows for locally quadratic convergence to the optimal trajectory.

Algorithm 1 outlines the propagation of the Discrete Euler-Lagrange (DEL) equations, enabling the efficient computation of optimal control inputs that account for the probabilistic evolution of the system. This iterative scheme is capable of handling the high-dimensional "expanded" state space created by the gPC expansion.

Numerical Validation and Results

The framework was validated on two simulated benchmarks to demonstrate its efficiency and robustness:

  • The Duffing Oscillator: A classic nonlinear system used to demonstrate how the algorithm handles complex, non-Gaussian probability distributions. The algorithm achieved locally quadratic convergence, reaching the optimal solution in roughly 12 iterations.
  • 12-DOF Quadrotor: A high-dimensional system where motor constants ($G_{tr}, G_{rot}$) were treated as uncertain and uniformly distributed. gPC-DDP successfully guided the vehicle to a target while keeping the "cloud" of possible trajectories tightly clustered, significantly reducing the standard deviation of the final state compared to standard DDP.

Furthermore, the Variational Integrator version (VI-gPC-DDP) was shown to be much more insensitive to the choice of time step ($\Delta t$) compared to standard Euler-based methods, allowing for successful planning where simpler methods failed.

Conclusion and Future Work

gPC-DDP offers a powerful model-based alternative for stochastic systems, providing fast convergence and physical accuracy. By tracking statistical moments explicitly, the robot can find the "most certain" path rather than just the fastest one.

However, challenges remain regarding the "curse of dimensionality" as the number of gPC coefficients grows exponentially with the number of uncertain parameters. Future work will explore sparse quadrature methods to handle higher-dimensional parameter spaces and multi-element gPC methods to maintain accuracy over very long time horizons where probability distributions become increasingly complex.

Summary updated

Abstract

In this paper we develop a novel, discrete-time optimal control framework for\nmechanical systems with uncertain model parameters. We consider finite-horizon\nproblems where the performance index depends on the statistical moments of the\nstochastic system. Our approach constitutes an extension of the original\nDifferential Dynamic Programming method and handles uncertainty through\ngeneralized Polynomial Chaos (gPC) theory. The developed iterative scheme is\ncapable of controlling the probabilistic evolution of the dynamic system and\ncan be used in planning and control. Moreover, its scalable and fast converging\nnature play a key role when dealing with complex, high-dimensional problems.\nBased on Lagrangian mechanics principles, we also prove that Variational\nIntegrators can be designed to properly propagate and linearize the gPC\nrepresentations of stochastic, forced mechanical systems in discrete time. We\nutilize this benefit to further improve the efficiency of our\ntrajectory-optimization methodology. Numerical simulations are included to\nvalidate the applicability of our approach.\n
BibTeX
@misc{boutselis2017stochastic,
  author = {George I. Boutselis and Yunpeng Pan and Gerardo De La Tore and Evangelos A. Theodorou},
  title = {{Stochastic trajectory optimization for mechanical systems with\textbackslash\{\}n parametric uncertainties}},
  year = {2017},
  eprint = {1705.05506},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1705.05506},
}
Figure