On the stability analysis of optimal state feedbacks as represented by deep neural models

Dario Izzo, Dharmesh Tailor, Thomas Vasileiou

Introduction

Deep neural networks, specifically Guidance and Control Networks (G&CNETs), have shown immense potential in approximating optimal control laws for complex dynamical systems. By imitating optimal controllers derived from classical methods, these networks offer near-instantaneous computation times suitable for real-time flight applications. However, their "black-box" nature remains a significant barrier to adoption in safety-critical aerospace and robotics domains. This research addresses this gap by introducing a framework to analyze the stability and robustness of neurocontrollers using linear stability analysis, time-delay margins, and high-order Taylor maps.

The G&CNET Architecture

To bridge the gap between deep learning and control theory, the authors utilize G&CNETs trained on optimal trajectories. These networks take state variables—such as position, velocity, and orientation—as inputs and output the necessary control actions.

G&CNET architecture for quadcopter control

A critical feature of these architectures is the use of smooth activation functions like softplus or tanh. This ensures the network is $C^\infty$ differentiable, allowing researchers to apply classical analytical tools like Jacobian linearization and automatic differentiation to evaluate the system's behavior at equilibrium points.

Analytical Methods for Stability

The study proposes three primary techniques to provide formal guarantees for neurocontrolled systems:

  1. Linear Stability Analysis: By calculating the Jacobian of the controller via automatic differentiation and combining it with the system's physics, the authors determine the eigenvalues at equilibrium. This confirms whether the system, such as a quadcopter in a hover state, will naturally return to stability after small perturbations.
  2. Stability to Time Delay: Recognizing that real-world hardware experiences communication and processing lag, the authors employ Padé approximations to model time-delayed systems. This allows for the calculation of a "critical time delay"—the maximum latency the system can tolerate before becoming unstable.
  3. High-Order Taylor Maps (HOTM): Utilizing differential algebra (via the pyaudi library), the system's trajectory is represented as a high-order Taylor polynomial. This method estimates the "convergence radius," defining a physical boundary around a path within which the controller is guaranteed to maintain stability despite initial condition errors.

Key Results and Insights

The framework was tested on a 2D quadcopter model, with G&CNETs trained on a dataset of 11.8 million state-action pairs derived from 200,000 optimal trajectories.

Root locus analysis of critical time delay

Several counterintuitive findings emerged:

  • Accuracy vs. Stability: Lower mean absolute error during training does not necessarily translate to better stability. In some cases, deeper and wider networks exhibited smaller stability margins than simpler architectures.
  • Robustness of Simplicity: Shallow networks (e.g., one hidden layer) often demonstrated significantly higher tolerance for time delays compared to deeper models, suggesting a trade-off between optimality and safety.
  • Quantifiable Safety: For the quadcopter model, critical time delays were calculated between 0.022s and 0.140s. Additionally, 7th-order Taylor maps estimated a convergence radius of approximately 0.1m for specific maneuvers, providing a concrete safety buffer for operations.

Conclusion

This work demonstrates that the "black box" of neural networks can be opened using established control theory tools. By treating DNNs as differentiable functions, engineers can derive rigorous stability metrics, such as lag tolerance and convergence radii. Future research aims to integrate these stability constraints directly into the neural network training process, fostering the development of controllers that are inherently both optimal and robust.

Summary updated

BibTeX
@article{izzo2021stability,
  author = {Dario Izzo and Dharmesh Tailor and Thomas Vasileiou},
  title = {{On the stability analysis of optimal state feedbacks as represented by deep neural models}},
  journal = {IEEE Trans. Aerosp. Electron. Syst.},
  year = {2021},
  doi = {10.1109/taes.2020.3010670},
}
Figure