Abstract
Verifying the safety of image-based Neural Network Control Systems (NNCS) is a critical challenge in autonomous robotics. Traditional formal verification methods struggle with high-dimensional image inputs because there is no direct mathematical model for real-world visual perception. This paper advances "surrogate verification"—using generative models like cGANs to simulate the environment—by introducing two key techniques: composition and unrolling. These methods mathematically fuse system physics with the neural network and delay simplification steps, drastically reducing overapproximation errors. The framework is successfully applied to verify safety in complex scenarios, including aircraft taxiing and emergency braking, across both CNN and Transformer architectures.
The Perception Gap in Autonomous Systems
Neural Network Control Systems (NNCS) are increasingly used for end-to-end control in autonomous vehicles. While tools exist to verify "state-based" systems (using low-dimensional inputs like velocity), image-based systems face a "perception gap." There is no precise formula to describe how a physical state—such as a car's distance from an obstacle—translates into the millions of pixels in a camera feed.
To address this, researchers use surrogate verification. A Conditional Generative Adversarial Network (cGAN) is trained to act as a proxy for the world, generating images based on the system's state. However, previous surrogate methods were often too conservative, suffering from "one-step errors" (losing the mathematical link between physics and AI) and "multi-step errors" (snowballing simplification errors over time).
Methodology: Composition and Unrolling
The authors propose two primary strategies to tighten safety bounds and make verification scalable:
1. Composition
Instead of analyzing the cGAN, the controller, and the system dynamics in isolation, the authors compose them into a single unified mathematical model. By treating the physics equations as an additional layer in the neural network or using "star-set" analysis to handle non-linearities, the verifier maintains the dependency between input states and control outputs. This ensures the system "knows" exactly how a physical change affects the final control signal, eliminating one-step errors.
2. Unrolling
To combat the accumulation of multi-step errors, the authors "unroll" multiple control steps into a single, larger neural network before performing an abstraction (the process of simplifying complex shapes into manageable mathematical boxes). By delaying this simplification, the verifier maintains a much higher degree of accuracy over time, preventing the "reachable set" (the area the robot might occupy) from expanding into unsafe zones unnecessarily.
Experimental Evaluation
The approach was evaluated on two high-stakes benchmarks:
- Autonomous Aircraft Taxiing: The system must keep an aircraft on a runway centerline using camera inputs. The new method produced a "converged reachable set" that was significantly tighter than previous baselines, proving the system is far less likely to drift than previously thought.
- Advanced Emergency Braking System (AEBS): A car must brake to avoid a collision. The authors tested both a deep Convolutional Neural Network (CNN) and a Transformer-based controller. While baseline methods failed to prove safety for any initial state, the proposed unrolling method successfully verified safety for both complex architectures.
Conclusion
By combining composition and unrolling, this research moves formal verification beyond "toy problems" and toward real-world autonomous architectures. While the computational cost remains high—requiring hundreds of GPU-hours for complex 3-step unrolling—the ability to provide safety guarantees for Transformer-based perception systems marks a significant milestone. Future work will focus on more advanced generative models, such as Video GANs, to better capture temporal dependencies in dynamic environments.
Summary updated