Learning an Approximate Model Predictive Controller with Guarantees

Michael Hertneck, Johannes Köhler, Sebastian Trimpe, Frank Allgöwer

Abstract

A supervised learning framework is proposed to approximate a model predictive controller (MPC) with reduced computational complexity and guarantees on stability and constraint satisfaction. The framework can be used for a wide class of nonlinear systems. Any standard supervised learning technique (e.g. neural networks) can be employed to approximate the MPC from samples. In order to obtain closed-loop guarantees for the learned MPC, a robust MPC design is combined with statistical learning bounds. The MPC design ensures robustness to inaccurate inputs within given bounds, and Hoeffding's Inequality is used to validate that the learned MPC satisfies these bounds with high confidence. The result is a closed-loop statistical guarantee on stability and constraint satisfaction for the learned MPC. The proposed learning-based MPC framework is illustrated on a nonlinear benchmark problem, for which we learn a neural network controller with guarantees.

Introduction

Model Predictive Control (MPC) is a premier strategy for managing complex nonlinear systems while adhering to strict safety constraints. By solving an optimization problem at each time step, MPC ensures optimal performance and safety. However, the computational cost of solving these optimizations online often limits its application in high-frequency systems or on hardware with limited resources.

While supervised learning offers a promising alternative by training neural networks to mimic MPC behavior, these "black-box" approximations typically lack formal safety guarantees. A minor approximation error could lead to system instability or constraint violations. This paper addresses this challenge by introducing a framework that combines the execution speed of neural networks with the rigorous mathematical safety of MPC.

Methodology

The proposed framework ensures that the learned controller inherits the robustness of the original MPC through a three-step process:

  1. Robust MPC (RMPC) Design: The foundation is a robust MPC designed to maintain stability despite small control input errors (denoted as $\eta$). This is achieved via constraint tightening, which creates a safety "buffer" to accommodate approximation inaccuracies.
  2. Supervised Learning: The RMPC is solved offline across a wide range of system states to generate a dataset (e.g., 1.6 million samples). A feedforward neural network is then trained on this data to map system states directly to control actions.
  3. Statistical Validation: To ensure the neural network's error remains within the RMPC's robustness margin, the authors utilize Hoeffding’s Inequality. By testing the controller on a large validation set, they derive a statistical bound that guarantees safety and stability with high confidence.
Overview of the proposed framework for learning an approximate MPC with statistical guarantees.

The framework was demonstrated on a Continuous Stirred Tank Reactor (CSTR) benchmark, a standard nonlinear problem in control theory.

Key Results and Insights

The experimental results highlight a significant leap in computational efficiency. The neural network-based Approximate MPC (AMPC) achieved an evaluation time of just 3 milliseconds, representing a 200x speedup over the 710 milliseconds required for the original RMPC optimization.

Comparison of state trajectories and control inputs between the learned Approximate MPC (AMPC) and a standard LQR.

In the CSTR example, the authors established with 99% confidence that the system would remain stable and satisfy all constraints for at least 99% of random initial conditions. Furthermore, the AMPC significantly outperformed a standard Linear Quadratic Regulator (LQR), which failed to stabilize the system under certain constraints where the AMPC remained safe and effective.

A core insight of this work is treating the neural network's approximation error as a bounded disturbance. By designing the original controller to be robust to such disturbances, the authors create a mathematical "space" for the neural network to operate safely. Shifting to statistical guarantees provides a practical path to certifying complex controllers for a high fraction of operating conditions with quantifiable confidence.

Discussion and Future Work

While the framework provides a robust path to high-speed control, it faces the "curse of dimensionality." Generating training data via state-space sampling becomes exponentially more difficult as system variables increase. Additionally, the guarantees are probabilistic; while sufficient for many industrial tasks, ultra-critical systems may require further deterministic checks. Future research will focus on improving sampling efficiency for higher-dimensional systems and exploring more sophisticated validation functions.

Summary updated

Abstract

A supervised learning framework is proposed to approximate a model predictive controller (MPC) with reduced computational complexity and guarantees on stability and constraint satisfaction. The framework can be used for a wide class of nonlinear systems. Any standard supervised learning technique (e.g. neural networks) can be employed to approximate the MPC from samples. In order to obtain closed-loop guarantees for the learned MPC, a robust MPC design is combined with statistical learning bounds. The MPC design ensures robustness to inaccurate inputs within given bounds, and Hoeffding's Inequality is used to validate that the learned MPC satisfies these bounds with high confidence. The result is a closed-loop statistical guarantee on stability and constraint satisfaction for the learned MPC. The proposed learning-based MPC framework is illustrated on a nonlinear benchmark problem, for which we learn a neural network controller with guarantees.
BibTeX
@inproceedings{hertneck2018learning,
  author = {Michael Hertneck and Johannes Köhler and Sebastian Trimpe and Frank Allgöwer},
  title = {{Learning an Approximate Model Predictive Controller with Guarantees}},
  booktitle = {IEEE Control. Syst. Lett},
  year = {2018},
  url = {https://openreview.net/forum?id=QictelPLyLs},
}
Figure