Combining Neural Networks and Symbolic Regression for Analytical Lyapunov Function Discovery

Jie Feng, Jie Feng 0006, Haohan Zou, Yuanyuan Shi

Abstract

Proving the stability of nonlinear dynamical systems is a fundamental challenge in control theory, traditionally requiring either rigid analytical templates or opaque "black-box" neural network certificates. We present CoNSAL (Combining Neural networks and Symbolic regression for Analytical Lyapunov function), a framework that bridges this gap. By distilling neural networks into precise mathematical formulas using symbolic regression, CoNSAL discovers analytical Lyapunov functions that are interpretable, easily verifiable, and robust. Our approach bypasses the verification bottleneck of deep learning models and achieves significantly higher success rates and faster verification times across various nonlinear benchmarks.

Introduction

In nonlinear dynamical systems, such as robotics or power grids, ensuring stability is critical. Engineers rely on Lyapunov functions—mathematical "energy" functions that decrease over time—to guarantee that a system will return to equilibrium. Historically, researchers faced a trade-off between classical methods (like Sum-of-Squares), which offer analytical clarity but lack flexibility, and Neural Lyapunov Functions, which are powerful but difficult to verify and interpret.

CoNSAL addresses this trade-off by leveraging the learning capacity of neural networks to find the underlying analytical formulas that scientists and engineers require for formal safety guarantees.

The CoNSAL Framework

The CoNSAL framework operates through an iterative process involving two nested loops:

  1. The Learner (Inner Loop): An Input-Convex Neural Network (ICNN) is trained to minimize "Lyapunov risk." It learns a candidate function $V(x)$ that satisfies stability conditions: $V(x) > 0$ for $x \neq 0$, $V(0) = 0$, and a negative Lie derivative $\dot{V}(x) < 0$, ensuring energy dissipation.
  2. The Falsifier (Outer Loop): Periodically, the framework employs Symbolic Regression (via the PySR package) to distill the neural network's learned shape into a closed-form mathematical expression.
The CoNSAL framework overview, illustrating the iterative loop between neural network learning, symbolic regression distillation, and counterexample-based falsification.

Instead of applying computationally expensive formal verification (like SMT solvers) directly to the neural network, CoNSAL performs numerical root-finding on the distilled analytical formula to identify "counterexamples"—points where stability conditions are violated. These points are then fed back into the training set to refine the neural network in the next iteration.

Compositional Design for Scalability

For high-dimensional systems, such as power grids, CoNSAL introduces a compositional approach. The system is decomposed into subsystems and edges, and the framework learns local Lyapunov functions that are aggregated into a joint certificate for the entire system.

Compositional design for high-dimensional systems, where a joint Lyapunov function is constructed from local neural network components.

Experimental Evaluation

The researchers evaluated CoNSAL on several nonlinear benchmarks:

  • Low-Dimensional Systems: 2-D linear path following, Inverted Pendulum, and Van Der Pol Oscillator; 3-D Trig dynamics; and 4-D Rotating Wheel Pendulum.
  • High-Dimensional Systems: A 6-D coupled nonlinear system and a 6-D 3-bus Power System to test compositional scalability.

Key Results

  • Improved Success Rates: On a 2-D path-following task, CoNSAL achieved an 80% success rate in finding a valid certificate, significantly outperforming traditional SMT-based neural network methods (30%).
  • Verification Speed: Distilling the network into a formula allowed for verification roughly 15x faster than using an SMT solver on the raw network (5.81s vs. 92.08s).
  • Scalability: The framework successfully managed networks with 128-neuron hidden layers—sizes that often cause formal verification tools to fail—because verification is performed on the simplified symbolic output.

Discussion and Future Work

CoNSAL demonstrates that symbolic regression can serve as more than just a post-processing step; it can be an active "teacher" during training. By converting complex networks into formulas, the framework bypasses the "verification bottleneck" of deep learning. In several instances, the algorithm rediscovered standard physical energy functions while also identifying novel local Lyapunov functions.

Current limitations include the difficulty symbolic regression faces in optimizing precise decimal constants and the reliance on numerical root-finding during training. Future work will focus on scaling the system beyond 10 dimensions and integrating it with simultaneous nonlinear controller design to provide stability-by-design guarantees.

Summary updated

BibTeX
@inproceedings{feng2024combining,
  author = {Jie Feng and Jie Feng and Haohan Zou and Yuanyuan Shi},
  title = {{Combining Neural Networks and Symbolic Regression for Analytical Lyapunov Function Discovery}},
  booktitle = {RLControlTheory @ ICML},
  year = {2024},
  url = {https://openreview.net/forum?id=Knj78wY9T4},
}
Figure