Mean-Field Stochastic Linear Quadratic Optimal Control for Jump-Diffusion Systems with Hybrid Disturbances

Chao Tang, Li Xue-qin, Qi WangSymmetry 2024 · DOI

TL;DR

This paper tackles a complex optimal control problem for systems with both continuous (Wiener process) and discrete (Poisson jumps) random disturbances, further complicated by "hybrid disturbances" in both the system dynamics and the cost function. The authors develop a novel framework to derive an explicit, implementable feedback control strategy by decoupling the system into Riccati equations and auxiliary differential equations, offering a more general solution than previous work.

What Problem Does This Paper Address?

Many real-world systems are subject to unpredictable random events. These can be continuous, like background noise (modeled by Wiener processes), or sudden, discrete events, like equipment failures or market crashes (modeled by Poisson jumps). This paper focuses on "jump-diffusion systems," which combine both types of randomness. Adding to the complexity, the authors introduce "hybrid disturbances" that affect not only the system's state but also the very objective function (cost) we're trying to minimize. This makes finding an optimal control strategy incredibly challenging. Existing methods often struggle with these combined disturbances and the "mean-field" aspect, where the system's behavior depends on the average behavior of many other similar systems. The paper aims to provide a robust mathematical framework for finding an optimal control policy in such intricate scenarios.

Methods

The core of the paper lies in developing a rigorous mathematical approach to solve this Mean-Field Stochastic Linear Quadratic (MF-SLQ) optimal control problem.

  • Well-posedness: They first establish that the jump-diffusion system itself is well-defined and has a unique solution using fixed-point theorems. This is a crucial first step before even considering control.
  • Optimality Conditions: Using variational calculus and duality techniques, they derive necessary and sufficient conditions for an optimal control. This involves introducing "adjoint processes" (similar to Lagrange multipliers in classical optimization) to link the control to the system's state and cost.
  • Decoupling Technique: The key innovation is a decoupling technique. They transform the complex system of forward-backward stochastic differential equations (MF-FBSDE) into a system of two Riccati differential equations and two auxiliary differential equations (one for the state and one for the disturbances). This is a significant simplification.
  • Riccati Equations: These Riccati equations are central to deriving the feedback control. Their solutions directly inform how the optimal control should adjust based on the current state of the system and its expected behavior.
  • Feedback Regulator: By solving these decoupled equations, they derive an explicit, implementable closed-loop feedback control strategy. This means the control action at any given time depends directly on the current system state and its expected future behavior, rather than requiring knowledge of the entire future trajectory.

Datasets

Not discussed in the paper. The paper focuses on theoretical development and provides a mathematical example rather than empirical evaluation on datasets.

Key Insights and Ideas

  • Unified Treatment of Disturbances: The paper successfully unifies the treatment of continuous (Wiener), discrete (Poisson jumps), and "hybrid" disturbances affecting both state and cost. This is a significant generalization.
  • Riccati Equations as the Key: The decoupling into Riccati equations is a powerful insight. It shows that even with complex mean-field interactions and hybrid disturbances, the core of the optimal control problem can often be reduced to solving these well-understood (though still complex) differential equations.
  • Implementable Feedback Control: The derivation of an explicit, closed-loop feedback control strategy is a major practical contribution. This moves beyond theoretical optimality conditions to a policy that can actually be implemented in a real system.
  • Connection to Stochastic Hamilton Systems: The paper establishes a clear link between the MF-SLQ problem and a corresponding mean-field stochastic Hamilton system, providing a deeper theoretical understanding of the problem's structure.

Key Results

  • Existence and Uniqueness: The paper proves the existence and uniqueness of solutions for the controlled jump-diffusion system and the MF-SLQ problem under specific assumptions.
  • Optimal Control Conditions: Necessary and sufficient conditions for optimality are derived, leading to a system of MF-FBSDEs.
  • Explicit Feedback Control: A closed-loop optimal control strategy is derived in an explicit form, dependent on the solutions of two Riccati differential equations and two perturbation differential equations.
  • Example Illustration: A one-dimensional example is provided to demonstrate the theoretical results, showcasing how the derived control strategy can be applied.

Limitations and Future Work

  • Deterministic Coefficients: The current work assumes deterministic coefficients for the system dynamics and cost function. The authors acknowledge that extending this to random coefficients is an open problem.
  • Indefinite Weighting Matrices: The paper's solvability relies on certain positive definiteness assumptions for the weighting matrices in the cost functional. Handling indefinite weighting matrices in the cost function remains a challenge.
  • Finite Horizon: The current analysis is for a finite time horizon. The authors suggest that considering the case of an infinite horizon would be a valuable future direction.
  • Generalization of Disturbances: While the paper handles hybrid disturbances, further exploration into more general forms of stochastic perturbations could be beneficial.

Summary updated

Abstract

A mean-field linear quadratic stochastic (MF-SLQ for short) optimal control problem with hybrid disturbances and cross terms in a finite horizon is concerned. The state equation is a systems driven by the Wiener process and the Poisson random martingale measure disturbed by some stochastic perturbations. The cost functional is also disturbed, which means more general cases could be characterized, especially when extra environment perturbations exist. In this paper, the well-posedness result on the jump diffusion systems is obtained by the fixed point theorem and also the solvability of the MF-SLQ problem. Actually, by virtue of adjoint variables, classic variational calculus, and some dual representation, an optimal condition is derived. Throughout our research, in order to connect the optimal control and the state directly, two Riccati differential equations, a BSDE with random jumps and an ordinary equation (ODE for short) on disturbance terms are obtained by a decoupling technique, which provide an optimal feedback regulator. Meanwhile, the relationship between the two Riccati equations and the so-called mean-field stochastic Hamilton system is established. Consequently, the optimal value is characterized by the initial state, disturbances, and original value of the Riccati equations. Finally, an example is provided to illustrate our theoretic results.
BibTeX
@inproceedings{tang2024mean,
  author = {Chao Tang and LI Xue-qin and Qi Wang},
  title = {{Mean-Field Stochastic Linear Quadratic Optimal Control for Jump-Diffusion Systems with Hybrid Disturbances}},
  booktitle = {Symmetry},
  year = {2024},
  doi = {10.3390/sym16060642},
}
Figure