Distributed Optimal Control and Application to Consensus of Multi-Agent Systems

Liping Zhang, Juanjuan Xu, Huanshui Zhang, Lihua XiearXiv 2023 · DOI · arXiv

Abstract: This paper develops a novel approach to the consensus problem of multi-agent systems by minimizing a weighted state error with neighbor agents via linear quadratic (LQ) optimal control theory. Existing consensus control algorithms only utilize the current state of each agent, and the design of distributed controller depends on nonzero eigenvalues of the communication topology. The presented optimal consensus controller is obtained by solving Riccati equations and designing appropriate observers to account for agents' historical state information. It is shown that the corresponding cost function under the proposed controllers is asymptotically optimal. Simulation examples demonstrate the effectiveness of the proposed scheme, and a much faster convergence speed than the conventional consensus methods. Moreover, the new method avoids computing nonzero eigenvalues of the communication topology as in the traditional consensus methods.

Introduction

In multi-agent systems (MAS)—such as swarms of drones, satellite formations, or autonomous vehicle fleets—achieving "consensus" is a fundamental requirement. Consensus is the process by which all agents agree on a specific variable (e.g., velocity or position) using only local communication. Traditional consensus protocols face two major hurdles: they are typically memoryless, ignoring historical state data, and they often require global knowledge of the communication graph's Laplacian eigenvalues for controller design. This global dependency is computationally expensive and contradicts the decentralized nature of MAS.

This paper introduces a distributed optimal control framework that addresses these limitations. By treating consensus as a Linear Quadratic (LQ) optimal control problem, agents minimize a cost function based on relative errors between themselves and their neighbors.

Methodology

The proposed approach integrates standard optimal control tools with distributed estimation. Key elements include:

  • Algebraic Riccati Equations (ARE): Feedback gains are derived to minimize the global error state.
  • Distributed Observers: Because agents lack a global view of the network, the authors designed observers for each agent to estimate the global error vector. These observers utilize both current neighbor interactions and historical state information.
  • Gain Optimization: Observer gains are optimized to minimize the spectral radius of the error system, ensuring rapid convergence.
Observer error trajectories for $\tilde{e}_1(k)$ demonstrating the convergence of the distributed estimation process.

The framework is also extended to support heterogeneous systems where agents may have different internal dynamics, moving beyond the common requirement for identical agents.

Results and Discussion

Simulation results validate the effectiveness of the proposed scheme. In a system of four agents, the proposed method reached consensus in approximately 12 steps, significantly faster than the 25 steps required by traditional state feedback methods.

State trajectories for $x_{i1}$ (top) and $x_{i2}$ (bottom) showing the agents reaching consensus over time.

Quantitative analysis confirms this speedup, with the closed-loop system exhibiting a lower spectral radius ($\rho = 0.7876$) compared to traditional methods ($\rho = 0.8777$). Furthermore, the authors provide rigorous mathematical proofs (Theorems 1-4) ensuring observer stability and asymptotic optimality—meaning the cost difference between this distributed controller and a theoretical centralized ideal vanishes over time.

Conclusion

The presented distributed optimal control framework offers a significant advancement in MAS coordination by improving convergence speed and removing the need for global eigenvalue calculations. While the current work assumes fixed topologies and reliable communication, future research will explore the impact of communication constraints like packet loss and time delays, as well as the extension to switching network topologies.

Summary updated

Abstract

This paper develops a novel approach to the consensus problem of multi-agent systems by minimizing a weighted state error with neighbor agents via linear quadratic (LQ) optimal control theory. Existing consensus control algorithms only utilize the current state of each agent, and the design of distributed controller depends on nonzero eigenvalues of the communication topology. The presented optimal consensus controller is obtained by solving Riccati equations and designing appropriate observers to account for agents' historical state information. It is shown that the corresponding cost function under the proposed controllers is asymptotically optimal. Simulation examples demonstrate the effectiveness of the proposed scheme, and a much faster convergence speed than the conventional consensus methods. Moreover, the new method avoids computing nonzero eigenvalues of the communication topology as in the traditional consensus methods.
BibTeX
@misc{zhang2023distributed,
  author = {Liping Zhang and Juanjuan Xu and Huanshui Zhang and Lihua Xie},
  title = {{Distributed Optimal Control and Application to Consensus of Multi-Agent Systems}},
  year = {2023},
  eprint = {2309.12577},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.12577},
}
Figure