Statistical Results on Filtering and Epi-convergence for Learning-Based Model Predictive Control

Anil Aswani, Humberto González, S. Shankar Sastry, Claire J. TomlinarXiv 2012 · arXiv

Abstract

Learning-Based Model Predictive Control (LBMPC) integrates traditional control theory with machine learning to enhance performance while maintaining safety. This paper provides the mathematical foundations for LBMPC, proving that nonparametric estimators can accurately identify unmodeled system dynamics. We establish that while simultaneous state estimation and model learning is generally impossible without full-state measurement, the resulting control laws exhibit epi-convergence to the true optimal control as data increases.

Introduction

In autonomous systems, a "nominal" model representing known physics is often augmented with "unmodeled dynamics" to account for complex effects like friction or aerodynamics. LBMPC utilizes a statistical "oracle" to learn these dynamics online. This work addresses two critical theoretical gaps: the observability problem (simultaneous state estimation and learning) and the convergence problem (ensuring control actions converge to the true optimum).

Methods

The theoretical framework utilizes several advanced mathematical tools to bridge control theory and statistics:

  • Popov-Belevitch-Hautus (PBH) Test: Employed to demonstrate the impossibility of simultaneous state and unmodeled dynamics estimation without full-state measurement.
  • Nonparametric Filtering: A local polynomial regression approach is proposed to estimate states without assuming a parametric form for errors, unlike standard Kalman filters.
  • L2-Regularized Nadaraya-Watson (L2NW) Estimator: This technique ensures the learned model is smooth and differentiable, facilitating its use in MPC optimization solvers.
  • Epi-convergence Theory: This framework analyzes the convergence of the minimizers of a sequence of functions, ensuring the resulting control actions are asymptotically optimal.

Key Insights

A primary finding is the "Observability Wall": if the output matrix is not full rank, it is mathematically impossible to distinguish internal states from constant offsets in unmodeled dynamics. This justifies why LBMPC typically assumes full-state measurement or high-fidelity estimation.

Furthermore, the authors identify a "kink" in the system trajectory. Because MPC control inputs are piecewise constant, changing at intervals of $T_u$, the state trajectory is once-differentiable but lacks second-order differentiability at the control update points.

Multi-rate sampling scheme showing control update period $T_u$ and measurement sampling period $T_s$.

To address this, a "left-side" and "right-side" filter design is required to handle discontinuities at the boundaries of the control intervals. Additionally, the system's trajectory must eventually pass near enough points in the state-space to "cover" the region of interest, a concept formalized as a Finite Sample Cover rather than through traditional ergodic theory.

Results

The paper proves the consistency of the proposed nonparametric filter with an error rate of $O_p(k^{-(r+1)/(2r+3)})$, where $k$ is the number of samples and $r$ is the polynomial order. The L2NW estimator is shown to be uniformly consistent, with error rates depending on regularization and bandwidth. Crucially, the LBMPC control law is proven to converge to the true optimal MPC law as the number of measurements approaches infinity.

Discussion and Limitations

While the requirement for full-state measurement and the computational cost of nonparametric methods remain challenges, the results solidify LBMPC as a mathematically rigorous approach to adaptive control. Future work may address the exploration-exploitation trade-off to ensure the system explores safely while satisfying the sufficient excitation requirements necessary for learning.

Summary updated

Abstract

Learning-based model predictive control (LBMPC) is a technique that provides deterministic guarantees on robustness, while statistical identification tools are used to identify richer models of the system in order to improve performance. This technical note provides proofs that elucidate the reasons for our choice of measurement model, as well as giving proofs concerning the stochastic convergence of LBMPC. The first part of this note discusses simultaneous state estimation and statistical identification (or learning) of unmodeled dynamics, for dynamical systems that can be described by ordinary differential equations (ODE's). The second part provides proofs concerning the epi-convergence of different statistical estimators that can be used with the learning-based model predictive control (LBMPC) technique. In particular, we prove results on the statistical properties of a nonparametric estimator that we have designed to have the correct deterministic and stochastic properties for numerical implementation when used in conjunction with LBMPC.
BibTeX
@misc{aswani2012statistical,
  author = {Anil Aswani and Humberto González and S. Shankar Sastry and Claire J. Tomlin},
  title = {{Statistical Results on Filtering and Epi-convergence for Learning-Based Model Predictive Control}},
  year = {2012},
  eprint = {1208.0864},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1208.0864},
}
Figure