Neural Lyapunov Control

Ya-Chien Chang, Nima Roohi, Sicun Gao

Abstract

We propose new methods for learning control policies and neural network Lyapunov functions for nonlinear control problems, with provable guarantee of stability. The framework consists of a learner that attempts to find the control and Lyapunov functions, and a falsifier that finds counterexamples to quickly guide the learner towards solutions. The procedure terminates when no counterexample is found by the falsifier, in which case the controlled nonlinear system is provably stable. The approach significantly simplifies the process of Lyapunov control design, provides end-to-end correctness guarantee, and can obtain much larger regions of attraction than existing methods such as LQR and SOS/SDP. We show experiments on how the new methods obtain high-quality solutions for challenging control problems.

Introduction

Controlling nonlinear systems—such as autonomous flight vehicles or multi-jointed robotic arms—presents a fundamental challenge in robotics. Traditional approaches typically rely on either local linearization (e.g., Linear Quadratic Regulators or LQR) or complex polynomial optimization (e.g., Sum-of-Squares or SOS). While LQR is efficient, it only guarantees stability in a small neighborhood near the equilibrium. Conversely, SOS methods provide formal guarantees but are computationally expensive and often produce conservative results that fail to capture the full potential of the system's dynamics.

This work bridges the gap between the expressive power of deep learning and the safety requirements of classical control theory by learning neural network Lyapunov functions that serve as formal stability certificates.

The Learner-Falsifier Framework

The core of our approach is a "Learner-Falsifier" loop that treats stability as an adversarial game.

  1. The Learner: A neural network with smooth activations (such as tanh) learns a control policy $u(x)$ and a Lyapunov function $V(x)$ simultaneously. It minimizes a "Lyapunov Risk" function, which penalizes states where the energy of the system fails to decrease over time.
  2. The Falsifier: An SMT-solver (dReal) searches the state space for counterexamples—points where the learner's stability proof is violated.
  3. The Loop: Counterexamples are fed back into the learner's training set, acting as an automated curriculum that focuses the network on the most difficult regions of the state space.
Training dynamics and performance: (a) trajectory comparison between LQR and NN, (b) Lyapunov risk convergence, and (c) falsifier-generated counterexamples guiding the learner.

As shown in the training dynamics above, the Lyapunov risk decreases as the learner incorporates counterexamples from the falsifier. Once the falsifier can no longer find any violations, the system is mathematically guaranteed to be stable within the verified domain.

Experimental Evaluation and Results

We evaluated our framework on several benchmark systems, including inverted pendulums, path-following kinematic models, and the Caltech Ducted Fan. The neural approach consistently outperformed traditional methods in both stability and the size of the "safe zone."

Expanding the Region of Attraction (ROA)

A key advantage of using neural networks is their ability to fit complex, non-convex Lyapunov landscapes. This flexibility allows the framework to identify much larger Regions of Attraction (ROA)—the set of initial states from which the controller is guaranteed to stabilize the system.

Visualization of learned Lyapunov functions (a-c) and the resulting expansion of the Region of Attraction (d) compared to traditional methods.

Our results demonstrate that the neural approach can obtain ROAs that are 300% to 600% larger than those found by LQR. Unlike SOS methods, which are often restricted to small, conservative ellipses, the Neural Lyapunov method finds larger, more accurate regions that reflect the true nonlinear limits of the system.

Conclusion

Neural Lyapunov Control provides a robust path toward provably safe autonomous systems. By combining the flexibility of deep learning with the rigor of formal verification, we can design controllers that are both highly performant and mathematically guaranteed to be stable. Future work will focus on scaling the falsifier to higher-dimensional systems and bridging the gap between simulation and physical hardware by accounting for sensor noise and model uncertainty.

Summary updated

Abstract

We propose new methods for learning control policies and neural network Lyapunov functions for nonlinear control problems, with provable guarantee of stability. The framework consists of a learner that attempts to find the control and Lyapunov functions, and a falsifier that finds counterexamples to quickly guide the learner towards solutions. The procedure terminates when no counterexample is found by the falsifier, in which case the controlled nonlinear system is provably stable. The approach significantly simplifies the process of Lyapunov control design, provides end-to-end correctness guarantee, and can obtain much larger regions of attraction than existing methods such as LQR and SOS/SDP. We show experiments on how the new methods obtain high-quality solutions for challenging control problems.
BibTeX
@inproceedings{chang2019neural,
  author = {Ya-Chien Chang and Nima Roohi and Sicun Gao},
  title = {{Neural Lyapunov Control}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=hgmEpERQ1U},
}
Figure