When deploying autonomous systems—like self-driving cars, drone swarms, or robotic patrollers—we face a fundamental tension between performance and safety. Machine learning (ML) excels at discovering highly optimized control policies for complex, high-dimensional tasks. However, deep neural networks and reinforcement learning agents are notoriously difficult to verify; they are black boxes that can fail catastrophically when encountering unfamiliar scenarios.
Conversely, traditional control theory offers formal verification methods like Hamilton-Jacobi reachability analysis, which provide ironclad mathematical guarantees that a system will remain safe. Yet, these classical methods suffer from two severe limitations: they require an exact, explicit mathematical model of the system's physics, and they suffer from an exponential "curse of dimensionality" that makes them computationally intractable for complex systems.
To bridge this gap, a powerful hybrid paradigm has emerged. By combining data-driven learning with formal methods, researchers are designing systems that use machine learning to navigate complex environments while relying on formal verification to guarantee absolute safety boundaries. As highlighted by Deshmukh et al. (2019) and Wongpiromsarn et al. (2023), this synergy allows formal specifications to guide machine learning components, yielding autonomous systems that are both highly capable and provably trustworthy.
Quantifying the Unknown with Gaussian Processes
The first major hurdle in verifying an autonomous system is that its exact physical dynamics are often unknown or too complex to model from first principles. If we do not have the exact differential equations governing a robot's movement, how can we mathematically prove it won't crash?
The solution lies in learning the dynamics from data using models that don't just predict the system's next state, but also quantify how uncertain they are about that prediction. Gaussian Process (GP) regression is the primary tool for this task. When a system collects input-output data from its environment, a GP constructs a model of the dynamics accompanied by a rigorous statistical confidence interval.
This explicit uncertainty quantification acts as the foundational bridge to formal verification. Instead of treating a learned model as absolute truth, verification frameworks factor the model's uncertainty directly into their safety calculations. If the robot operates in a well-explored region of its state space, the GP's uncertainty is low, allowing the robot more freedom. If it approaches an unvisited or highly volatile region, the uncertainty bounds expand, signaling the verification engine to act conservatively.
Mechanism 1: Discretization via Interval Markov Models
To run formal verification algorithms on a continuous, uncertain system learned via GPs, researchers often translate the continuous model into a discrete, symbolic abstraction.
In a standard abstraction, a continuous state space is carved into a grid of discrete states, and transitions between these states are assigned precise probabilities. However, because our data-driven GP model contains inherent uncertainty, we cannot know the exact transition probabilities.
To resolve this, Jackson et al. (2020) and Skovbekk et al. (2025) pioneered frameworks that abstract unknown dynamical systems into Interval Markov Decision Processes (IMDPs) or Interval Markov Chains (IMCs). Instead of calculating a single probability for transitioning from State $A$ to State $B$, the GP's uncertainty bounds are used to calculate a lower and upper bound—an interval of probabilities (e.g., $[0.65, 0.80]$).
[ Continuous State Space ]
│
▼ (GP Regression + Uncertainty Bounds)
[ Interval Markov Abstraction ] ──► Transition: [0.65, 0.80]
│
▼ (Value Iteration / Model Checking)
[ Robust Control Strategy ] ──► Guaranteed Safety Envelope
When a formal verification algorithm reasons over an IMDP, it evaluates strategies against a adversarial "worst-case" selection from within those intervals. Jackson et al. (2021) demonstrated that this approach allows for the synthesis of robust control strategies even within partially-known, switched stochastic systems. If an agent enters an unexplored region where data is sparse, the transition intervals widen to $[0, 1]$, forcing the verifier to assume the worst possible outcome and steering the agent back to safety.
To prevent this grid-based abstraction from exploding computationally, Skovbekk et al. (2023) introduced a method that partitions the noise domain rather than just the state space. By clustering complex, non-standard noise distributions, they construct tight interval abstractions without requiring dense grid refinements, mitigating the curse of dimensionality.
Mechanism 2: Continuous Safety Filters and Certificates
Discretizing the state space is highly effective for strategic planning, but some applications require continuous, real-time safety enforcement. For these scenarios, researchers combine machine learning with continuous mathematical "shields" known as safety certificates.
Control Barrier Functions
One prominent type of certificate is the Control Barrier Function (CBF). A CBF mathematically defines a safe region of the state space and acts as a continuous filter on the system's actuators. As long as the autonomous agent operates safely inside the designated zone, the underlying machine learning policy is given full control to optimize performance or explore the environment. However, if the agent approaches the boundary of the safe set, the CBF overrides the ML policy, applying the minimum corrective action necessary to keep the system safe.
When the system dynamics are unknown, Jagtap et al. (2020) showed that GPs can be used to learn the unknown components of the nonlinear system while the CBF is modified to absorb the GP's worst-case prediction errors. This guarantees, with high probability, that the physical system will never breach its boundaries, effectively separating the task of learning to perform from the task of guaranteeing safety.
Scalable Reachability Learning
Another approach adapts Hamilton-Jacobi (HJ) reachability, which computes "reach-avoid" sets—the exact boundaries of states from which a system can safely reach a goal while avoiding obstacles. While classical HJ reachability is limited to low-dimensional systems due to computational constraints (Tomlin et al., 2018), recent work leverages deep learning to scale these certificates.
Li et al. (2025) introduced a framework that uses deep reinforcement learning to approximate these reach-avoid value functions for high-dimensional systems. By enforcing a novel Lipschitz-continuous structure on the learned value function and pairing it with Second-Order Cone Programming (SOCP), the authors provide deterministic safety certificates for complex trajectories, validating the approach on high-speed drone racing and highway driving simulations.
Similarly, Bharadwaj et al. (2019) demonstrated that embedding formal reachability constraints within learning-enabled visibility optimization allows autonomous agents to successfully execute complex surveillance-evasion games without risking a loss of target tracking.
From Theory to Software Ecosystems
As these hybrid formal-ML approaches mature, the focus is expanding from theoretical proofs to accessible software implementations. Transitioning from manual mathematical derivations to automated verification pipeline tools is essential for practical engineering.
A key step in this direction is markovml, a Python package developed by Maaz et al. (2025). The package allows developers to build Markov processes where complex, data-driven parameters are directly informed or learned by embedded machine learning models. Rather than relying on imprecise, computationally expensive Monte Carlo simulations to test if a system is safe, markovml enables users to compute mathematically rigorous, formal verification bounds directly over the learning-enabled architecture.
By embedding machine learning within formal abstractions and wrapping black-box policies in rigorous safety certificates, this research direction provides a viable path forward for autonomous systems: leveraging the adaptive power of AI without sacrificing the absolute guarantees of formal mathematics.
Key Foundational Work
Specification Formalisms for Modern Cyber-Physical Systems (Dagstuhl Seminar 19071) (2019) — contributed 2 concept elements to this direction.
Hamilton-Jacobi Reachability: Some Recent Theoretical Advances and Applications in Unmanned Airspace Management (2018) — contributed 1 concept elements to this direction.
Strategy Synthesis for Surveillance-Evasion Games with Learning-Enabled Visibility Optimization (2019) — contributed 1 concept elements to this direction.