Learning an Approximate Model Predictive Controller With Guarantees

Michael Hertneck, Johannes Köhler, Sebastian Trimpe, Frank Allgöwer

TL;DR

This paper introduces a framework to replace computationally expensive nonlinear Model Predictive Controllers (MPC) with fast, learned approximations (like neural networks) without sacrificing safety. By combining robust control design with statistical validation via Hoeffding’s Inequality, the authors provide rigorous closed-loop guarantees for stability and constraint satisfaction for the learned controller.

What Problem Does This Paper Address?

Model Predictive Control (MPC) is a gold standard for controlling complex systems because it can handle nonlinearities and strict safety constraints by solving an optimization problem at every time step. However, solving these optimizations "online" is computationally heavy, making MPC difficult to use on cheap hardware or for systems requiring very high sampling rates (like drones or fast robotics).

While researchers have tried to "mimic" MPC using supervised learning (training a neural network to act like the controller), these "black-box" approximations usually lose the formal safety guarantees of the original MPC. A small approximation error in the neural network could lead to a catastrophic system failure or a violation of critical safety boundaries. This paper aims to bridge that gap: providing the speed of a neural network with the mathematical safety of MPC.

Methods

The authors propose a three-step framework:

  1. Robust MPC (RMPC) Design: Instead of a standard MPC, they design a "Robust" version that is mathematically guaranteed to remain stable even if there is a small error ($\eta$) in the control input. They achieve this using constraint tightening, essentially making the safety boundaries slightly stricter than necessary to create a "buffer" for errors.
  2. Supervised Learning: The RMPC is solved offline for millions of different system states to create a dataset. A function approximator (in this case, a feedforward Neural Network) is trained on this data to map system states directly to control actions.
  3. Statistical Validation: To ensure the learned controller stays within the "buffer" created in Step 1, the authors use Hoeffding’s Inequality. By testing the learned controller on a large validation dataset, they calculate a statistical bound. This allows them to state, with high confidence (e.g., 99%), that the approximation error is small enough to maintain system stability and safety.

Datasets

The paper evaluates the framework using a simulated Continuous Stirred Tank Reactor (CSTR), a standard nonlinear benchmark in control theory.

  • Training Data: 1.6 million samples were generated by solving the RMPC over a uniform grid of the system's state space.
  • Validation Data: 34,980 trajectories were used to statistically verify the approximation error and ensure the guarantees held true.

Key Insights and Ideas

  • The Error Margin as a Bridge: The most elegant idea here is treating the approximation error of the neural network as a "disturbance." By designing the original controller to be robust to disturbances, the authors create a mathematical "space" where the neural network can operate safely even if it isn't a 100% perfect replica.
  • Statistical vs. Deterministic Guarantees: Proving a neural network is safe for every possible state is notoriously difficult (the "reachability" problem). By shifting to a statistical guarantee, the authors provide a practical way to certify controllers for a high fraction of initial conditions with a quantifiable confidence level.
  • Automation of Design: The framework provides a clear recipe (Algorithm 1) for engineers to follow, moving from system requirements to a certified, high-speed controller.

Key Results

  • Massive Speedup: The neural network controller was evaluated in just 3 milliseconds, compared to 710 milliseconds for the original RMPC optimization. This represents a 200x speedup, making it viable for real-time applications that the original MPC could not handle.
  • Certified Safety: In the CSTR example, the authors proved with 99% confidence that the system would remain stable and satisfy all constraints for at least 99% of random initial conditions.
  • Superior Performance: The learned Approximate MPC (AMPC) significantly outperformed a standard Linear Quadratic Regulator (LQR). While the LQR failed to stabilize the system under certain constraints, the AMPC remained safe and effective.

Limitations and Future Work

  • Curse of Dimensionality: Generating the training data requires sampling the state space. For systems with many variables (high-dimensional states), the number of samples needed grows exponentially, which could make the offline training phase prohibitively slow.
  • Probabilistic Nature: The guarantees are statistical, not absolute. While a 99.9% guarantee is sufficient for many industrial tasks, it may not be enough for ultra-critical "human-in-the-loop" systems like commercial avionics without further deterministic checks.
  • Future Directions: The authors suggest exploring more sophisticated "indicator functions" for validation and applying the framework to higher-dimensional systems where sampling efficiency becomes a major hurdle.

Summary updated

Abstract

A supervised learning framework is proposed to approximate a model predictive controller (MPC) with reduced computational complexity and guarantees on stability and constraint satisfaction. The framework can be used for a wide class of nonlinear systems. Any standard supervised learning technique (e.g., neural networks) can be employed to approximate the MPC from samples. In order to obtain closed-loop guarantees for the learned MPC, a robust MPC design is combined with statistical learning bounds. The MPC design ensures robustness to inaccurate inputs within given bounds, and Hoeffding's Inequality is used to validate that the learned MPC satisfies these bounds with high confidence. The result is a closed-loop statistical guarantee on stability and constraint satisfaction for the learned MPC. The proposed learning-based MPC framework is illustrated on a nonlinear benchmark problem, for which we learn a neural network controller with guarantees.
BibTeX
@inproceedings{hertneck2018learning,
  author = {Michael Hertneck and Johannes Köhler and Sebastian Trimpe and Frank Allgöwer},
  title = {{Learning an Approximate Model Predictive Controller With Guarantees}},
  booktitle = {IEEE Control. Syst. Lett},
  year = {2018},
  url = {https://openreview.net/forum?id=QictelPLyLs},
}

Loading related content...

Figure