Learning-Based Event-Triggered MPC With Gaussian Processes Under Terminal Constraints

Kazumune Hashimoto, Yuga Onoue, Akifumi Wachi, Xun ShenIEEE Trans. Cybernetics 2021 · DOI · OpenReview

TL;DR

This paper presents a new approach to Model Predictive Control (MPC) for systems with unknown dynamics, using Gaussian Processes (GPs) to learn the system's behavior. It introduces an "event-triggered" mechanism that significantly reduces computational load by only re-solving complex control problems when necessary, ensuring stability and finite-time convergence to a safe state.

What Problem Does This Paper Address?

Model Predictive Control (MPC) is a powerful technique for optimizing system performance while respecting constraints. However, it's computationally intensive, especially for complex, nonlinear systems. Traditional MPC requires solving an optimization problem at every time step. Event-triggered MPC aims to reduce this computational burden by only re-solving the optimization problem when a specific "event" occurs, typically when the system's actual state deviates significantly from its predicted state.

A key challenge in applying MPC, especially event-triggered MPC, to real-world systems is that their dynamics are often unknown beforehand. While learning-based approaches, particularly those using Gaussian Processes (GPs), can learn these unknown dynamics, integrating them into an event-triggered MPC framework with guaranteed stability and efficient computation remains an open problem. Specifically, ensuring that the system remains stable and reaches a desired "terminal set" (a safe operating region) in finite time, while minimizing computational effort, is the core challenge addressed here.

Methods

The paper proposes a Learning-based Event-Triggered MPC (ET-MPC) framework. Here's a breakdown of the key methods:

  • Gaussian Process (GP) Regression for Dynamics Learning: The unknown nonlinear dynamics of the system are learned using GP regression. This allows the MPC to predict future system states based on past data. The paper uses a squared exponential (SE) kernel for the GP.
  • Terminal Constraints and Symbolic Models: To ensure stability and convergence, a terminal constraint is introduced. This means the system must reach a predefined "terminal set" by the end of the prediction horizon. The paper uses a novel approach involving symbolic models and set-invariant theory to construct this terminal set, which can be non-convex.
  • Event-Triggered Condition based on Recursive Feasibility: The core of the ET-MPC is the event-triggered condition. This condition determines when the optimal control problem (OCP) needs to be re-solved. It's derived based on the concept of "recursive feasibility," ensuring that if the current OCP is solvable, the next one will also be solvable. Specifically, the OCP is re-solved only when the error between the predicted state (from the GP) and the actual state exceeds a certain threshold.
  • Decreasing Horizon Strategy: To further improve efficiency and convergence, a decreasing prediction horizon strategy is employed. As the system progresses and the GP model becomes more accurate, the prediction horizon of the OCP is reduced.
  • Iterative Task Algorithm: A two-phase algorithm is proposed: an "update phase" to learn the system dynamics and construct the terminal set, and an "execution phase" to implement the ET-MPC and collect new data. This iterative process refines the GP model and improves performance over time.

Datasets

  • Training Data: The paper utilizes training data (input-output pairs) to train the Gaussian Process models for learning the system dynamics. The specific source or nature of this initial training data is not detailed beyond mentioning it can be obtained from an expert or a model-free controller.
  • Evaluation Data: The performance of the proposed ET-MPC is evaluated through a numerical simulation of a leader-follower tracking control problem for a nonholonomic mobile robot. The simulation uses a perturbed system model derived from the robot's dynamics.

Key Insights and Ideas

  • Bridging Learning and Event-Triggering: The paper successfully integrates GP-based learning of unknown dynamics with an event-triggered MPC framework, offering a way to achieve efficient control for complex systems without prior model knowledge.
  • Guaranteed Stability via Recursive Feasibility: Deriving the event-triggered condition from recursive feasibility is a key insight that provides theoretical guarantees for the solvability of future control problems, crucial for maintaining stability.
  • Symbolic Models for Robust Terminal Set Construction: The use of symbolic models to construct a terminal set is a novel contribution, allowing for more flexible and potentially non-convex safe regions, which can be more practical for complex systems.
  • Iterative Improvement: The proposed iterative algorithm allows the system to learn and adapt over time, improving its control performance and reducing computational load as more data is collected.
  • Computational Efficiency: The event-triggered nature, coupled with the decreasing horizon strategy, significantly reduces the number of OCP solves compared to traditional periodic MPC, especially as the learning process matures.

Key Results

  • Finite-Time Convergence: The proposed ET-MPC algorithm is theoretically shown to guarantee that the system state enters the terminal set in finite time, provided the uncertainty in the GP model becomes sufficiently small.
  • Recursive Feasibility: The paper proves that the optimal control problem remains recursively feasible under the proposed event-triggered condition.
  • Numerical Simulation: A simulation of a nonholonomic mobile robot tracking problem demonstrates the effectiveness of the approach.
  • The number of OCP solves (triggers) significantly decreases over iterations.
  • The total computation time is substantially reduced compared to periodic MPC, especially after several iterations. For instance, at iteration 10, the proposed method's computation time was 74 ms, while periodic MPC took 1073 ms.
  • The tracking performance improves over iterations, with the robot achieving better accuracy in following the reference trajectory.

Limitations and Future Work

  • GP Uncertainty Assumption: The theoretical analysis relies on the assumption that the uncertainty in the GP model becomes "small enough." The paper acknowledges that achieving this might require sufficient training data and careful selection of GP hyperparameters.
  • SE Kernel Restriction: The theoretical analysis is primarily based on the squared exponential (SE) kernel. While SE kernels are powerful, extending the results to other kernel types could broaden applicability.
  • Initial Data Acquisition: The method requires an initial training dataset. The paper suggests obtaining this from an expert or a model-free controller, but efficient and robust methods for initial data acquisition could be a future research direction.
  • Computational Cost of GP: While the ET-MPC reduces OCP solves, the GP regression itself can become computationally intensive with a large number of data points. Future work could explore more efficient GP implementations or alternative learning methods.
  • Extension to Other System Types: The paper focuses on discrete-time nonlinear systems with additive noise. Extending the framework to continuous-time systems, systems with multiplicative noise, or more complex disturbances could be valuable.

Summary updated

Abstract

The event-triggered control strategy is capable of significantly reducing the number of control task executions while achieving desired control objectives, such as stability. In this article, we introduce a novel learning-based method for event-triggered model predictive control with initially unknown dynamics. The formulation of optimal control problems (OCPs) is based on predictive states derived from Gaussian process (GP) regression under terminal constraints. The event-triggered condition proposed in this article is derived from the recursive feasibility, so that the OCPs are solved only when an error between the predictive and the actual states exceeds a certain threshold. This article analyzes the convergence of the closed-loop system under the event-triggered condition, demonstrating that the system’s state will enter the terminal set within a finite time, assuming small-enough uncertainty in the GP model. We validate this approach through a tracking control problem, illustrating its practical effectiveness.
BibTeX
@article{hashimoto2021learning,
  author = {Kazumune Hashimoto and Yuga Onoue and Akifumi Wachi and Xun Shen},
  title = {{Learning-Based Event-Triggered MPC With Gaussian Processes Under Terminal Constraints}},
  journal = {IEEE Trans. Cybernetics},
  year = {2021},
  doi = {10.48550/arxiv.2110.12214},
}
Figure