Deep neural networks are exceptional at learning complex, high-dimensional functions, making them highly attractive for controlling nonlinear dynamical systems like autonomous quadrotors, robotic arms, and power grids. However, standard neural networks are notorious "black boxes." A single out-of-distribution input or an unpredictable oscillation during training can cause a neural controller to output a catastrophic action, causing the physical system to diverge or crash.
To prevent this, early approaches tried adding soft penalties to the loss function during training to encourage stable behavior, a strategy explored by Yu et al.. However, as noted by Suykens et al., soft constraints cannot provide absolute mathematical guarantees; if the optimization process encounters a loophole or an unseen state, the system can still fail.
This limitation has driven a paradigm shift toward stability-by-design. Instead of checking for stability after training or hoping a loss function enforces it, researchers are embedding control-theoretic constraints directly into the neural network's architecture. By restricting the mathematical space of what the network can compute, these models are structurally incapable of causing instability.
Shaping the Energy Landscape with Neural Lyapunov Certificates
To understand how architectural stability works, we must look to classical control theory's most powerful tool: the Lyapunov function.
Imagine a bowl with a marble rolling inside it. No matter where you release the marble, it will eventually roll down to the bottom and stop. In control theory, the bowl represents a Lyapunov function $V(x)$—a mathematical representation of the system's energy. The bottom of the bowl is the target equilibrium state. If we can prove that a system's state always moves down this energy landscape ($\dot{V}(x) < 0$), we guarantee that the system is stable and will safely converge to its goal.
Instead of guessing a polynomial formula for this bowl, we can use a neural network to learn it. Crucially, we must structurally force the network to look like a valid bowl. Berkenkamp et al. demonstrated that embedding these mathematical properties directly into a neural network's architecture ensures that its output remains a valid Lyapunov candidate by design throughout the entire learning process.
To make this work, the network must satisfy two core properties: it must be strictly positive everywhere except at the goal, and it must have a unique global minimum at that goal. Yang achieved this by integrating a learned 1-norm term into the function's architecture, preventing the network from collapsing into trivial or invalid flat landscapes near the equilibrium point.
Standard Network (Unconstrained) Input-Convex Network (Lyapunov Bound)
_ _ \ /
/ \ <- Local Minima \ / <- Guaranteed Unique
___/ \___ (System Trapped) \_/ Global Minimum
To enforce the "bowl" shape over a wide area, Mukherjee et al. leveraged Input-Convex Neural Networks (ICNNs). By constraining the network's weights to be non-negative and using convex activation functions, the network is structurally incapable of forming "fake" local minima or peaks. This allows the model to capture highly complex, non-convex safety boundaries while guaranteeing a single, stable convergence point—a significant leap over the restrictive, rigid quadratic templates traditionally used by authors like Lutter et al..
Controllers as Gradients of Convex Functions
Learning a stable energy landscape is only half the battle; we also need a control policy that actively guides the system down that landscape. An elegant way to guarantee this is to structure the controller itself using monotonicity.
In physics, conservative forces (like gravity pulling an object down a hill) are represented as the gradient of a potential function. We can mimic this by parameterizing a neural network controller as the gradient of a strictly convex neural potential function. As shown by Cui et al., enforcing weight positivity and specific activation constraints forces the neural network to act as a Lyapunov function itself. Because the controller outputs actions that strictly follow the gradient of this convex landscape, the closed-loop system is guaranteed to be dissipative—meaning it naturally sheds excess energy and returns to equilibrium.
This architectural constraint eliminates the chaotic, unpredictable actions typical of standard deep reinforcement learning. For instance, Jiang et al. utilized monotonic neural network architectures to design frequency controllers for electrical power grids. Because the policy is structurally constrained to be monotonic, it is mathematically guaranteed never to overcorrect or trigger a cascading blackout, even when handling highly volatile renewable energy inputs.
Differentiable Projections as Mathematical Guardrails
What if we want to use a highly expressive, completely unconstrained neural network architecture to maximize performance, but still require absolute safety? We can achieve this by placing a differentiable projection layer at the very end of the network.
Think of a projection layer as a mathematical guardrail. The core neural network can guess any control action it wants. If the action is safe, it passes through unchanged. If the action would cause the system to diverge, the projection layer instantly intercepts it and maps it to the closest mathematically stable action on a safe manifold.
[ Nominal Neural Network ] --> Unsafe Action --> [ Differentiable Projection ] --> Safe Action
|
(Snaps action to safe
manifold boundary)
Kojima implemented this by designing a differentiable layer that projects nominal network outputs onto a manifold satisfying the Hamilton-Jacobi Inequality, bounding the system's output energy relative to its input energy. Similarly, Yang et al. integrated analytical projection operators directly into neural control frameworks to guarantee exponential stability by design.
Because these projection operations are fully analytical and differentiable, the entire system can still be trained end-to-end using standard backpropagation. If the network proposes an unstable action, the gradient signal flowing back through the projection layer penalizes the network, teaching it to find high-performance solutions that naturally lie within the safe boundaries.
Incremental Stability via Contraction Theory
A final, powerful perspective shifts the focus from converging to a fixed point (Lyapunov stability) to converging toward a desired trajectory. This is governed by contraction theory, which analyzes whether two neighboring trajectories naturally draw closer together over time.
If a system is contracting, any disturbance or tracking error will exponentially decay, pulling the system back onto its intended path. Song et al. demonstrated that by embedding contraction-theoretic constraints directly into a neural network's architecture, complex nonlinear stability conditions can be transformed into simple linear constraints on the network's gradients.
This structural approach ensures that the network remains stable throughout the entire training process and deployment, even when environmental parameters shift unexpectedly. By shifting stability from an optimization penalty to an architectural requirement, these frameworks allow deep learning to be safely deployed in the most critical, high-stakes physical systems.
Key Foundational Work
NLq theory: unifications in the theory of neural networks, systems and control (1995) — contributed 3 concept elements to this direction.
Neural Control for Nonlinear Dynamic Systems (1995) — contributed 2 concept elements to this direction.