Stochastic Adaptive Linear Quadratic Differential Games

Nian Liu, Lei GuoOpenReview 2026 · OpenReview

TL;DR

This paper tackles the challenge of designing adaptive strategies for two-player zero-sum linear-quadratic differential games where the system's dynamics are unknown. The authors propose a novel approach that combines weighted least squares estimation with a diminishing excitation technique to ensure global stability and convergence to a Nash equilibrium, even with uncertain parameters.

What Problem Does This Paper Address?

Differential games are powerful tools for modeling interactions in complex systems, from economics to engineering. A common scenario is the linear-quadratic (LQ) differential game, which offers a tractable framework. However, most existing work assumes that the system's parameters (like matrices A, B1, B2, D in the state dynamics) are known to all players. This is a significant limitation, as real-world systems often have unknown or uncertain dynamics.

This paper addresses this gap by focusing on stochastic adaptive LQ differential games where these crucial system matrices are unknown to both players. This necessitates the development of adaptive strategies – strategies that can learn and adjust based on observed system behavior. The challenge lies in designing these adaptive strategies such that the game remains stable and players can still reach a desirable outcome, specifically a Nash equilibrium, despite the ongoing learning process.

Methods

The authors' approach is built upon several key components:

  • Weighted Least Squares (WLS) Estimation: To handle unknown system parameters, they employ a WLS estimation algorithm. This algorithm continuously updates estimates of the system matrices based on observed states and control inputs. A crucial aspect is ensuring the self-convergence of these estimates, meaning they converge to the true parameters over time.
  • Random Regularization: To guarantee that the estimated system models are always "controllable" (a necessary condition for stability and solvability of related equations), a random regularization procedure is introduced. This technique helps to create a uniformly controllable family of estimated models, preventing the system from becoming unstable due to poor parameter estimates.
  • Diminishing Excitation Technique: To ensure that the adaptive strategies not only stabilize the system but also drive the parameter estimates towards their true values (strong consistency), a diminishing excitation method is used. This involves carefully injecting small, controlled perturbations into the system's dynamics, which aids in the learning process without destabilizing the game.
  • Adaptive Strategies: The core of the solution involves constructing feedback strategies for both players. These strategies are designed to be implementable using the online parameter estimates. The paper considers two cases for strategy design, depending on whether the algebraic Riccati equation (ARE) associated with the game has a specific type of solution.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Bridging Adaptive Control and Game Theory: The paper successfully extends the well-developed theory of adaptive control (dealing with unknown system parameters) to the domain of differential games. This is a significant step towards analyzing more realistic, uncertain multi-agent systems.
  • Guaranteed Stability and Convergence: A major achievement is demonstrating that even with unknown parameters and ongoing adaptation, the closed-loop game system can be made globally stable. Furthermore, the system is shown to asymptotically converge to a Nash equilibrium.
  • The Role of Regularization: The introduction of random regularization is key to overcoming a common hurdle in adaptive control: ensuring the controllability of estimated models. This technique provides a robust way to maintain desirable system properties during adaptation.
  • Diminishing Excitation for Strong Consistency: The paper highlights the importance of carefully managed excitation for achieving strong consistency of parameter estimates. This is crucial for ensuring that the adaptive strategies eventually behave like optimal strategies for the true system.

Key Results

The paper presents three main theorems:

  1. Global Stability: The adaptive strategies guarantee that the closed-loop game system is globally stable, meaning the state remains bounded for any initial condition.
  2. Parameter Convergence: The WLS parameter estimates are shown to converge to the true system parameters almost surely as time goes to infinity.
  3. Nash Equilibrium: Under certain conditions (controllability and antianalytic prefactorizability of the system's transfer function), the proposed adaptive strategies constitute a feedback Nash equilibrium. The resulting game value is also characterized.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Time-Varying Parameters: The current work assumes constant, albeit unknown, system parameters. Extending the theory to handle time-varying parameters would be a significant advancement.
  • Asymmetric Information: The paper assumes common knowledge of the system matrices (though unknown). Investigating adaptive games where players have different or asymmetric information structures is a natural next step.
  • Generalizations: The authors suggest that similar results might be obtainable for more general multi-player games and under different structural conditions.
  • Global Regulator: The possibility of a "global regulator" that influences all players is mentioned as an interesting avenue for further research.

Summary updated

Abstract

Game theory is playing more and more important roles in understanding complex systems and in investigating intelligent machines with various uncertainties. As a starting point, we consider the classical two-player zero-sum linear-quadratic stochastic differential games, but in contrast to most of the existing studies, the coefficient matrices of the systems are assumed to be unknown to both players, and consequently it is necessary to study adaptive strategies of the players, which may be termed as adaptive games and which has rarely been explored in the literature. In this article, by introducing a suitable information structure for adaptive games, we will show that a theory can be established on adaptive strategies that are designed based on both the certainty equivalence principle and the diminishing excitation technique. Under almost the same physical structure conditions as those in the traditional known parameters case, it is shown that the closed-loop adaptive game systems will be globally stable and asymptotically reach the Nash equilibrium.
BibTeX
@misc{liu2026stochastic,
  author = {NIAN LIU and Lei Guo},
  title = {{Stochastic Adaptive Linear Quadratic Differential Games}},
  year = {2026},
  url = {https://openreview.net/forum?id=rXgZRVZy1G},
  howpublished = {OpenReview},
}
Figure