Learning and Verifying Stabilizing Controllers with Structured Neural Control Lyapunov Functions

Jose Luis Ledesma Martin, Patricia PauliarXiv 2026 · arXiv

TL;DR

This paper introduces a framework for jointly learning neural network controllers and Control Lyapunov Functions (CLFs) with formal stability guarantees for nonlinear systems. By utilizing structured architectures like Polyak-Łojasiewicz Networks (PLNets), the authors ensure critical stability properties by design, enabling an iterative verification process that significantly expands the certified "safe" operating region of the controller.

What Problem Does This Paper Address?

Controlling nonlinear systems (like a swinging pendulum or a self-driving car) is challenging because small errors can lead to instability. While neural networks are excellent at approximating complex control laws, they are often "black boxes" lacking safety guarantees.

A standard way to prove stability is using a Control Lyapunov Function (CLF)—a mathematical "energy" function that must be positive everywhere except at the goal and must strictly decrease as the system moves. Finding a valid CLF and a corresponding controller simultaneously is a "chicken-and-egg" problem. Furthermore, verifying that these conditions hold over an entire continuous region (not just at a few test points) usually requires computationally expensive formal methods.

Methods

The authors propose a three-pronged approach to bridge the gap between neural network expressivity and formal safety:

  1. Structured Architectures: Instead of standard "flat" neural networks, they use Input Convex Neural Networks (ICNNs) and Polyak-Łojasiewicz Networks (PLNets). These architectures are mathematically constrained to be positive definite "by construction." This means the verifier doesn't have to check if the function is positive; it only has to check if it decreases over time.
  2. Joint Training Strategy: They employ a multi-phase training pipeline:
  3. Phase 1: The controller learns to imitate a simple linear (LQR) controller.
  4. Phase 2: The CLF is "warmed up" to fit the initial controller.
  5. Phase 3: Both are trained together using Counterexample-Guided Synthesis, where the system specifically looks for states that violate stability and retrains on them.
  6. Formal Verification & ROA Expansion: They use Mixed-Integer Linear Programming (MILP) and Mixed-Integer Quadratic Programming (MIQP) to formally prove stability. Crucially, they introduce an iterative Region of Attraction (ROA) expansion algorithm. It starts with a small verified box and "grows" it by verifying successive rings around the origin, maximizing the certified safe area without needing to retrain the models.
Fig. 1: Result of the ROA expansion procedure. The certified region is incrementally expanded by verifying rings around the original training region X, yielding a larger certified ROA

Datasets

The methods were evaluated on two classic nonlinear control benchmarks:

  • Inverted Pendulum: A classic underactuated system where the goal is to balance a pole upright.
  • Path-Tracking System: A kinematic model of a vehicle following a circular reference path.

The authors compared their structured CLFs against standard Multi-Layer Perceptrons (MLPs) and traditional quadratic baselines.

Key Insights and Ideas

  • Structure is a Cheat Code: By baking the "positivity" requirement into the network architecture (especially with PLNets), the authors drastically reduced the search space for the optimizer and the workload for the formal verifier.
  • The Power of PLNets: Unlike ICNNs, which only provide a lower bound, PLNets provide both lower and upper quadratic bounds. This "sandwiching" allows for much better control over the geometry of the stable region.
  • Lipschitz Tuning: The ratio between the network's monotonicity ($\mu$) and Lipschitz ($\nu$) constants acts as a "knob" for the shape of the stable region. Smaller ratios lead to more "circular" (isotropic) stable regions, which often result in larger certified areas.
Fig. 2: Comparison of certified ROA for the inverted pen- dulum across PLNet, ICNN-L2, MLP and quadratic CLF architectures, showing the best performing seed for each.

Key Results

  • Superior Coverage: In the inverted pendulum task, the PLNet achieved a certified ROA nearly double the size of the next best neural architecture (ICNN-L1).
  • Verification Efficiency: PLNets were remarkably consistent, often requiring only a single round of certification to prove stability over the initial training domain, whereas other architectures struggled with multiple rounds of counterexample generation.
  • Expansion Success: The iterative expansion algorithm successfully grew the certified regions far beyond the initial training boxes. For the path-tracking system, the quadratic baseline generalized best to expanded regions, but the PLNet provided the largest initial certified area among neural methods.
Fig. 4: Effect of the Lipschitz bound ν on the certified region of attraction for the inverted pendulum, with µ = 0.5 fixed.

Limitations and Future Work

  • Scalability: The formal verification step relies on MILP/MIQP solvers (like CPLEX), which can struggle as the state-space dimensionality increases. The current experiments were limited to 2D systems.
  • Expansion Strategy: The current ROA expansion uses a simple fixed-step "ring" approach. The authors suggest that more sophisticated, adaptive expansion strategies could yield even larger certified regions.
  • Discrete-Time Focus: The work focuses on discrete-time systems; extending these structured guarantees to continuous-time neural ODE-based controllers remains an open area for exploration.

Summary updated

Abstract

This work investigates the joint learning of neural controllers and control Lyapunov functions (CLFs) for nonlinear discrete-time systems. We exploit the structural properties of Input Convex Neural Networks and Polyak-Łojasiewicz Networks to guarantee positive definiteness of CLF candidates by construction, reducing the conditions requiring formal verification. We develop a mixed-integer programming framework to formally verify the remaining stability conditions and introduce an iterative region of attraction (ROA) expansion procedure to enlarge the certified ROA without retraining. Experiments on an inverted pendulum and a path-tracking system demonstrate the advantages of structured neural CLFs, particularly PLNets, and illustrate how their structural bounds influence the geometry of the certified regions. Our code is available at: https://github.com/JLedesMart/Structured-Neural-Control-Lyapunov-Functions.
BibTeX
@misc{martin2026learning,
  author = {Jose Luis Ledesma Martin and Patricia Pauli},
  title = {{Learning and Verifying Stabilizing Controllers with Structured Neural Control Lyapunov Functions}},
  year = {2026},
  eprint = {2611.21009},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21009},
}
Figure