Traditional formal verification—the mathematical proof that a system will never enter an unsafe state—is built for low-dimensional systems. It works beautifully for a drone described by a 6-variable state vector (position and velocity), but it collapses when faced with a single 1024x1024 image. The "curse of dimensionality" makes calculating safety boundaries in pixel space computationally impossible.
To bridge this gap, researchers are moving the "safety logic" out of the high-dimensional pixel space and into compressed, low-dimensional latent manifolds. By verifying the system within these learned representations, we can provide formal guarantees for robots that "see" the world through cameras.
The Dimensionality Bottleneck
Formal safety methods like Hamilton-Jacobi (HJ) reachability or Control Barrier Functions (CBFs) require solving complex differential equations over the entire state space. In a 3D world, this is manageable. In a pixel world, where every pixel is a dimension, the search space is infinite.
The core insight across recent work is that while images are high-dimensional, the underlying factors that matter for safety (e.g., the distance to a wall, the angle of a runway) are low-dimensional. Sydney M. Katz and Mykel Kochenderfer demonstrated this in Verification of Image-based Neural Network Controllers Using Generative Models. Instead of verifying the controller against every possible image, they used a Generative Adversarial Network (GAN) to map a low-dimensional state space to realistic images. By concatenating this generator with the controller, they created a verifiable pipeline where the safety proof happens on the low-dimensional "latent" input to the GAN rather than the pixels themselves.
Learning Safe Latent Dynamics
If we don't have a pre-defined state space, we can learn one. This is the approach of "World Models," where an agent learns a compressed representation of its environment.
In CERL: Safe Exploration Using Bayesian World Models and Log-Barrier Optimization, Yarden As and Andreas Krause show that safety constraints can be enforced directly within this learned latent space. By using a Bayesian world model, the agent can account for uncertainty; if the agent is unsure about the safety of a latent transition, the "log-barrier" optimizer prevents it from taking that action. This allows robots to learn complex tasks, like visual navigation, without a single catastrophic failure during training.
Similarly, the SAGE framework by Bingqing Chen and Sylvia Lee Herbert scales HJ reachability to autonomous racing. They use a Variational Autoencoder (VAE) to compress ego-vision images into a latent state, then compute a "safety value function" on that embedding. This allows a racing agent to push physical limits because it has a mathematically grounded "safety critic" watching its latent trajectory.
Prioritizing Separation over Reconstruction
A common pitfall in latent space modeling is trying to reconstruct the image perfectly. However, for safety, we don't need to know the color of the wallpaper; we only need to know where the wall is.
Manan Tayal and Shishir Kolathaya introduced the SaLaD (Safety-based Latent Dynamics) model, which explicitly prioritizes the separation of safe and unsafe regions in the latent space. By focusing the encoder on features relevant to hazard avoidance rather than high-fidelity reconstruction, they make the task of defining Control Barrier Certificates much simpler and more robust.
This concept of "structured" latent spaces is further explored in LS3 by Albert Wilcox and Ken Goldberg. They use "Latent Space Safe Sets" to ensure that every planned action sequence ends in a region where the robot is confident it can succeed. This "reach-avoid" logic—ensuring you can always return to a known safe state—is what enables long-horizon tasks like surgical cable routing to be performed safely from pixels.
Scaling Verification with Proof Sharing
Even in latent space, verifying every single frame can be slow. Christian Sprecher and Marc Fischer proposed a "shortcut" in Shared Certificates for Neural Network Verification. They realized that many inputs map to similar regions in the latent space. By creating "reusable" safety certificates—mathematical templates that cover entire neighborhoods of the latent space—they can skip the expensive verification process for new inputs that fall within already-certified regions.
Why It Works: The Intuition
The success of this research direction stems from a fundamental realization: Safety is a low-bandwidth signal. While a camera provides megabytes of data per second, the "safety state" of the system (Am I about to hit something? Is the temperature too high?) is usually just a few numbers.
By using generative models and encoders as a "perception filter," researchers like Hussein Sibai and Yiannis Kantaros in their SPVT framework are able to decouple the complexity of the world from the complexity of the safety proof. The generative model handles the messy, high-dimensional reality, while the formal verification handles the clean, low-dimensional logic. This division of labor is what finally makes provably safe vision-based robotics possible.