Stochastic economic model predictive control for Markovian switching systems

Pantelis Sopasakis, Domagoj Herceg, Panagiotis Patrinos, Alberto Bemporad

TL;DR

This paper extends Economic Model Predictive Control (EMPC) to stochastic systems that switch between different operating modes, known as Markovian switching systems. The authors provide the first rigorous theoretical framework to guarantee that these controllers remain feasible over time, achieve bounded average performance, and maintain mean-square stability.

What Problem Does This Paper Address?

Traditional Model Predictive Control (MPC) is designed to make a system follow a specific "set-point" or trajectory. In contrast, Economic MPC (EMPC) throws away the set-point and directly optimizes a performance metric, such as minimizing fuel consumption or maximizing profit.

While EMPC has been well-studied for deterministic systems (where everything is predictable), real-world engineering systems are often plagued by uncertainty. Specifically, many systems switch between different "modes" of operation—think of a power grid switching between energy sources or a chemical reactor changing phases. These are modeled as Markovian switching systems.

The challenge is that when you optimize for economics in a switching environment, standard stability proofs break down. There was a significant theoretical gap: how do we ensure a stochastic EMPC controller won't eventually lead to an infeasible state (a system crash) or fail to settle into a stable regime?

Methods

The authors tackle the problem by proposing two main EMPC formulations for nonlinear Markovian switching systems:

  1. Terminal Equality Constraints: They introduce the concept of a "bet node"—the most likely successor mode in the Markov chain. By forcing the system's predicted terminal state to match the optimal steady state of this "bet node," they ensure the controller can always find a valid move in the next time step.
  2. Terminal Regions and Penalty Functions: For a more flexible approach, they use terminal sets ($X_f$) and penalty functions ($V_f$). They provide a method to design these using local linearization and Bilinear Matrix Inequalities (BMIs).

The "secret sauce" for their stability proofs is Stochastic Dissipativity. They use a "rotated cost function" technique. By mathematically transforming the economic cost into a "rotated" version that looks like a traditional tracking cost, they can use Lyapunov-style arguments to prove stability without actually changing the optimal control actions.

Datasets

Not discussed in the paper. This is a purely theoretical paper focused on providing mathematical proofs and design guidelines. No numerical simulations or external datasets were used.

Key Insights and Ideas

  • The "Bet Node" Strategy: To handle the branching paths of a Markov process, the authors suggest anchoring the terminal constraint to the most probable future mode. This provides a balance between being prepared for uncertainty and maintaining a solvable optimization problem.
  • Performance Guarantees: The paper proves that the EMPC-controlled system will have an asymptotic average performance that is at least as good as the best possible steady-state operation.
  • The Power of Rotation: The authors show that even if an economic cost function is non-quadratic or non-convex, it can be analyzed through the lens of dissipativity. This allows them to prove Mean Square Stability (MSS), meaning the system state doesn't just stay bounded but actually converges in a statistical sense.
  • Bridging Linear and Nonlinear: While the core theory handles nonlinear systems, the authors provide a practical bridge by showing how to design controllers for nonlinear systems using local linear approximations.

Key Results

  • Recursive Feasibility: The authors mathematically proved that if the EMPC problem is feasible at the start, it will remain feasible for all future time steps, despite the random mode switching.
  • Stability Criteria: They established sufficient conditions for Mean Square Stability (MSS) under both terminal equality and terminal region constraints.
  • Design Framework: The paper provides a concrete computational path for engineers. By solving a set of matrix inequalities (specifically BMIs), one can determine the terminal costs and regions needed to guarantee a stable, economically optimal controller.

Limitations and Future Work

  • Conservatism of the "Bet Node": While the "bet node" approach ensures feasibility, it might be conservative if the Markov transition probabilities are widely distributed (i.e., if there isn't one clearly "most likely" next mode).
  • Local vs. Global Stability: The design guidelines for terminal regions rely on local linearization. This means the stability guarantees are "local"—they work as long as the system stays within a certain neighborhood of the optimal equilibrium.
  • Computational Complexity: Solving BMIs for terminal region design is more difficult than solving standard Linear Matrix Inequalities (LMIs). While the authors suggest a line-search method, this could become a bottleneck for very high-dimensional systems.
  • Future Directions: The authors suggest that future work could involve relaxing the "common equilibrium" assumption (Assumption 7) to better handle systems where different modes have vastly different optimal operating points.

Summary updated

Abstract

The optimization of process economics within the model predictive control (MPC) formulation has given rise to a new control paradigm known as economic MPC (EMPC). Several authors have discussed the closed-loop properties of EMPC-controlled deterministic systems, however, little have uncertain systems been studied. In this paper we propose EMPC formulations for nonlinear Markovian switching systems which guarantee recursive feasibility, asymptotic performance bounds and constrained mean square (MS) stability.
BibTeX
@inproceedings{sopasakis2017stochastic,
  author = {Pantelis Sopasakis and Domagoj Herceg and Panagiotis Patrinos and Alberto Bemporad},
  title = {{Stochastic economic model predictive control for Markovian switching systems}},
  booktitle = {IFAC-PapersOnLine},
  year = {2017},
  doi = {10.1016/j.ifacol.2017.08.056},
}

Loading related content...

Figure