Optimality Conditions for Model Predictive Control: Rethinking Predictive Model Design

Akhil S Anand, Arash Bahari Kordabad, Mario Zanon, Sébastien GrosarXiv 2024 · DOI · arXiv

TL;DR

This paper challenges the conventional wisdom that the best Model Predictive Control (MPC) comes from the most accurate predictive model. By deriving the necessary and sufficient mathematical conditions for closed-loop optimality, the authors show that a model doesn't need to "fit the data" perfectly; instead, it must satisfy specific inequalities that link the model's dynamics directly to the system's cost function and value function.

What Problem Does This Paper Address?

In Model Predictive Control (MPC), we use a mathematical model to predict future system behavior and choose the best sequence of actions. Naturally, the standard engineering approach is to make that model as accurate as possible using System Identification (SYSID) or Machine Learning (ML). We typically minimize the "prediction error"—the gap between what the model says will happen and what actually happens.

However, there is a known "gap" in control theory: a model with low prediction error doesn't always result in a high-performance controller, especially in "Economic MPC" where the goal is to minimize a specific cost (like fuel or time) rather than just following a reference path. In stochastic systems, a model that captures the average behavior might lead to suboptimal decisions. The authors tackle a fundamental question: What exactly must a model satisfy to ensure the resulting MPC policy is truly optimal for the real system?

Methods

The authors move away from simple "curve fitting" and instead use the rigorous framework of Markov Decision Processes (MDPs) and Bellman equations.

Key technical components include:

  • Advantage Functions: They define the "optimality" of a model based on the difference between the Action-Value function ($Q$) and the Value function ($V$).
  • Class $\mathcal{K}$ Functions: They use these strictly increasing functions to create bounds. For a model to be "optimal," its internal "advantage" must be bounded by the true system's "advantage."
  • Theorem 1 (The Core Contribution): The authors derive a necessary and sufficient condition (an inequality) that a model-based MDP must satisfy to yield the same optimal policy as the true system.
  • Deterministic MPC Mapping: They translate these broad MDP conditions into the specific context of deterministic MPC, which is the most common form used in industry.

Datasets

Not discussed in the paper. This is a theoretical paper focused on establishing mathematical proofs and conditions. It does not include empirical testing on specific datasets or benchmarks.

Key Insights and Ideas

  • The "Best Fit" is not the "Best Control": The most surprising insight is that the conditions for optimality are often counterintuitive to traditional data-fitting. A model can be "wrong" in its predictions but "right" in its control actions if it preserves the correct ordering of actions in the eyes of the cost function.
  • Cost-Aware Modeling: The model's requirements are intrinsically linked to the stage cost ($L$) and the true value function ($V^*$). You cannot design an optimal model without considering what you are trying to optimize.
  • Single-Step Sufficiency: Interestingly, the conditions only need to apply to the single-step predictions of the model. If the model is "good enough" for one step in a way that respects the system's value function, the entire multi-step MPC horizon will align with the optimal policy.
  • Flexibility in Design: There isn't just one "optimal model." The paper shows there is a whole class of models that satisfy these conditions, giving engineers more freedom to simplify models as long as they stay within the derived bounds.

Key Results

The paper provides a mathematical foundation for Performance-Oriented System Identification.

  • Theorem 1 establishes the necessary and sufficient inequality: the model-based advantage function must be "sandwiched" between two scaled versions of the true system's advantage function.
  • Corollary 2 simplifies this for specific cases, showing that an MPC scheme can achieve optimality even with model inaccuracies if the stage cost is appropriately adjusted (a concept known as "cost-tuning").
  • Corollary 3 provides the specific version of these conditions for deterministic MPC, which is the "bread and butter" of industrial control.

Limitations and Future Work

The primary limitation is that the paper is purely theoretical. While it tells us what the conditions are, it doesn't provide a ready-to-use algorithm for how to construct such a model from raw data.

The authors suggest that the next logical step is to turn these necessary and sufficient conditions into loss functions for Machine Learning. Instead of training a neural network to minimize prediction error (MSE), one could train it to satisfy the inequalities derived in Theorem 1. This would lead to "control-aware" models that are purpose-built for optimality rather than just accuracy.

Summary updated

Abstract

Optimality is a critical aspect of Model Predictive Control (MPC), especially in economic MPC. However, achieving optimality in MPC presents significant challenges, and may even be impossible, due to inherent inaccuracies in the predictive models. Predictive models often fail to accurately capture the true system dynamics, such as in the presence of stochasticity, leading to suboptimal MPC policies. In this paper, we establish the necessary and sufficient conditions on the underlying prediction model for an MPC scheme to achieve closed-loop optimality. Interestingly, these conditions are counterintuitive to the traditional approach of building predictive models that best fit the data. These conditions present a mathematical foundation for constructing models that are directly linked to the performance of the resulting MPC scheme.
BibTeX
@misc{anand2024optimality,
  author = {Akhil S Anand and Arash Bahari Kordabad and Mario Zanon and Sébastien Gros},
  title = {{Optimality Conditions for Model Predictive Control: Rethinking Predictive Model Design}},
  year = {2024},
  eprint = {2412.18268},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.18268},
}

Loading related content...

Figure