Abstract
This article is concerned with stability and performance of controlled stochastic processes under receding horizon policies. We carry out a systematic study of methods to guarantee stability under receding horizon policies via appropriate selections of cost functions in the underlying finite-horizon optimal control problem. We also obtain quantitative bounds on the performance of the system under receding horizon policies as measured by the long-run expected average cost. The results are illustrated with the help of several simple examples.
Introduction
Receding Horizon Control (RHC), commonly known as Model Predictive Control (MPC), has established itself as a premier strategy for deterministic systems. By solving a finite-horizon optimization problem at each time step and executing only the first action, RHC provides a computationally feasible approximation of infinite-horizon control. In deterministic settings, stability is typically ensured by imposing terminal costs and terminal set constraints.
However, these methods often falter when applied to stochastic systems. In the presence of unbounded noise (such as Gaussian disturbances), the system state can be pushed out of any predefined "invariant" terminal set with non-zero probability. This paper bridges the gap between control theory and the theory of Markov processes to provide a rigorous framework for the stability and performance of stochastic predictive control without relying on restrictive hard constraints.
Methodology: Stability via Drift Conditions
The authors propose a systematic approach to stability based on Cost Function Selection. Rather than adding complex constraints to the optimization problem, they demonstrate that stability emerges naturally when the stage cost $c$ and the terminal cost $c_F$ are chosen to satisfy a negative drift condition.
By leveraging Markov stability theory, the paper treats the closed-loop system as a Markov chain. The core idea is to show that the optimal value function $V^*_N$ from the $N$-step optimization problem acts as a Lyapunov function. If the expected value of this cost function decreases (drifts downward) whenever the state is outside a specific "safe" compact set, the system is guaranteed to be stable in a mean-square or geometric sense.
Key Theoretical Pillars
- Theorem 3: Establishes the relationship between the finite-horizon optimal value function and the drift condition required for stability.
- Theorem 11: Provides a concrete upper bound on the long-run expected average cost. This theorem is pivotal as it links the mathematical "drift" constant to a tangible performance metric, allowing engineers to bound the expected error or fuel consumption over an infinite timeline.
Analytical Examples and Simulations
The framework was evaluated using several benchmark control problems to demonstrate its versatility:
- Linear Quadratic (LQ) Problems: The authors show how the framework applies to standard linear systems with quadratic costs.
- Systems with Bounded Controls: A critical real-world scenario where actuators have physical limits (saturation). The paper proves that even with saturated controls, geometric stability is achievable if the noise is not too "heavy-tailed."
- Indicator Cost Functions: Demonstrating stability when costs are used to penalize the state for exiting a desired operating range.
Key Insights
- Stability as Statistical Pull: In a stochastic context, stability is not about staying within a fixed boundary, but about ensuring a statistical "pull" back toward the origin.
- Performance-Stability Link: There is a direct mathematical duality between proving stability and bounding performance. The same drift condition used to prove the system won't diverge also provides the upper bound for the long-run average cost.
- Robustness to Noise: For systems with Gaussian noise and bounded controls, the authors proved that the probability of the state wandering far from the origin decays exponentially, representing a very high degree of stochastic stability.
Limitations and Future Directions
While the paper provides a robust theoretical foundation, it notes several areas for further development:
- Computational Complexity: The paper focuses on the properties of the optimal policy rather than the numerical methods required to find it. Solving stochastic optimization problems in real-time remains a significant challenge.
- Cost Design: Practitioners may find it challenging to design cost functions that satisfy the necessary mathematical drift conditions while still reflecting the physical goals of the system.
- Case-Specific Parameters: Determining the exact drift constants and compact sets for complex, high-dimensional systems requires detailed case-by-case analysis.
Conclusion
This research provides a recipe for designing stable stochastic controllers by shifting the focus from hard constraints to the intelligent design of cost functions. By integrating Lyapunov-like drift conditions into the RHC framework, the authors offer a systematic way to guarantee both the stability and the long-term performance of systems operating under uncertainty.
Summary updated