Deep neural networks and reinforcement learning (RL) agents excel at finding highly performant control policies for complex, high-dimensional systems. However, their black-box nature presents a fundamental challenge for safety-critical applications like autonomous driving, robotics, and industrial process control. Traditional control theory relies on rigorous mathematical proofs to guarantee that a system will remain stable and never enter an unsafe state. In contrast, purely data-driven models offer empirical success but lack these deterministic guarantees, occasionally failing catastrophically when encountering edge cases outside their training distribution.
To bridge this gap, a major research direction focuses on combining the expressive power of neural networks with the mathematical rigor of formal verification methods. The goal is to create "correct-by-construction" or formally verifiable neural control loops.
Neural Certificates for Stability and Safety
In classical control, safety and stability are verified using certificates—scalar functions over the state space that act as mathematical guarantees of system behavior. The two most common types are Lyapunov functions (which prove a system will converge to a stable equilibrium) and Control Barrier Functions (CBFs) (which prove a system will remain within a designated safe region, a property known as forward invariance).
For a function $V(x)$ to be a valid Lyapunov function, it must satisfy specific conditions, such as being positive definite ($V(x) > 0$ for all states $x \neq 0$) and having a negative derivative along the system's trajectories ($\dot{V}(x) < 0$). Finding these functions analytically for complex, nonlinear systems is notoriously difficult.
The core insight of recent work is to use neural networks to learn both the control policy and its corresponding safety certificate simultaneously. For instance, Nima Roohi in Neural Lyapunov Control demonstrated that deep networks can approximate these stability functions for complex nonlinear dynamics.
However, learning an approximation is not enough; it must be formally verified. To achieve absolute certainty, researchers couple the neural network training with Satisfiability Modulo Theories (SMT) solvers or formal logic tools. The solver acts as an adversary, searching the state space for any point where the learned neural certificate violates the required mathematical conditions. If a violation is found, it is fed back into the training loop as a counterexample. This loop continues until the solver can mathematically prove that no violations exist.
This approach has been extended to safety boundaries. Andrea Peruffo and Daniele Ahmed developed methods for the Automated and Formal Synthesis of Neural Barrier Certificates for Dynamical Models, providing a framework to synthesize provably safe boundaries. To maximize the certified operating space, Jun Liu et al. proposed using physics-informed neural networks to solve Zubov's partial differential equation, allowing the verification of much larger Maximal Neural Lyapunov Functions.
Runtime Enforcement via Neurosymbolic Shielding
While verifying a neural network globally provides the strongest guarantees, it can be computationally prohibitive for highly dynamic or complex environments. An alternative approach is runtime enforcement, commonly implemented as a safety "shield."
Instead of trying to prove that the neural network will never choose an unsafe action, a shield acts as an online supervisor. It intercepts the action proposed by the unverified neural network policy, evaluates its safety using a predictive or symbolic model of the environment, and overrides it if it threatens to violate safety constraints.
As surveyed by Bettina Könighofer et al. in Correct-by-Construction Runtime Enforcement in AI - A Survey, this architecture decouples the complex task of optimizing performance from the strict requirement of ensuring safety. The reinforcement learning agent is free to explore and optimize its policy within the safe boundaries enforced by the shield.
In Shields for Safe Reinforcement Learning, Bettina Könighofer and Nils Jansen explain how this neurosymbolic combination guarantees zero catastrophic failures during both the training and deployment phases. To mitigate the computational overhead of continuous online monitoring, Davide Corsi et al. introduced Verification-Guided Shielding for Deep Reinforcement Learning, which partitions the state space offline into provably safe and potentially unsafe regions, activating the runtime shield only when the agent approaches a dangerous boundary.
Reachability Analysis and Probabilistic Relaxations
When systems scale to multiple interacting agents or high-dimensional state spaces, exact formal verification faces the curse of dimensionality. To handle these scenarios, researchers employ reachability analysis and statistical verification frameworks.
Reachability analysis computes the set of all states a system can possibly reach over a time horizon given a set of initial conditions and uncertainties. For multi-agent systems, computing these sets globally is incredibly difficult. To solve this, Shayegan Omidshafiei et al. developed a framework called ReBAR (Collision Avoidance Verification of Multiagent Systems with Learned Policies), which shifts the analysis into a relative coordinate frame. By performing pairwise backward reachability checks, they can formally verify collision avoidance for scaling numbers of agents.
When absolute deterministic verification is mathematically impossible due to stochastic environments or unmodeled dynamics, researchers pivot to rigorous probabilistic guarantees. Instead of proving a failure probability of exactly zero, they establish strict statistical bounds. For example, Albert Lin 0003 and Somil Bansal in Verification of neural reachable tubes via scenario optimization and conformal prediction established a mathematical equivalence between scenario optimization and conformal prediction. This allows them to compute "outlier-adjusted" reachable tubes, recovering a larger safe operating space while maintaining rigorous, distribution-free probabilistic safety bounds.