Abstract
Bayesian neural networks (BNNs) retain NN structures with a probability distribution placed over their weights. With the introduced uncertainties and redundancies, BNNs are proper choices of robust controllers for safety-critical control systems. This paper considers the problem of verifying the safety of nonlinear closed-loop systems with BNN controllers over unbounded-time horizon. In essence, we compute a safe weight set such that as long as the BNN controller is always applied with weights sampled from the safe weight set, the controlled system is guaranteed to be safe. We propose a novel two-phase method for the safe weight set computation. First, we construct a reference safe control set that constraints the control inputs, through polynomial approximation to the BNN controller followed by polynomial-optimization-based barrier certificate generation. Then, the computation of safe weight set is reduced to a range inclusion problem of the BNN on the system domain w.r.t. the safe control set, which can be solved incrementally and the set of safe weights can be extracted. Compared with the existing method based on invariant learning and mixed-integer linear programming, we could compute safe weight sets with larger radii on a series of linear benchmarks. Moreover, experiments on a series of widely used nonlinear control tasks show that our method can synthesize large safe weight sets with probability measure as high as 95% even for a large-scale system of dimension 7.
Introduction
Deep Neural Networks (DNNs) are powerful controllers but often operate as "black boxes," behaving unpredictably when encountering out-of-distribution data. Bayesian Neural Networks (BNNs) address this by placing a probability distribution over their weights, allowing them to express uncertainty. However, verifying the safety of a BNN-controlled system is significantly more complex than verifying a deterministic DNN, as a BNN represents a distribution of possible networks.
Existing verification methods often rely on discretizing continuous systems and using Mixed-Integer Linear Programming (MILP), which struggles to scale and frequently results in overly conservative (small) safe regions. This research introduces a framework to verify safety for continuous, nonlinear systems without discretization, aiming for larger, more robust safe weight sets.
Methodology
The proposed approach utilizes a two-phase framework to bridge the gap between BNN parameter uncertainty and system-level safety guarantees.
Phase 1: Constructing a Reference Safe Control Set
The first phase establishes a deterministic baseline. The "mean" weight of the BNN is used to create a reference controller, which is then approximated by a polynomial. Using Sum-of-Squares (SOS) optimization, the researchers generate a Barrier Certificate—a mathematical function that proves the system's trajectory cannot enter unsafe regions. This phase defines a "safe control set," representing the range of allowable control inputs ($u$) that maintain system safety.
Phase 2: Incremental Safe Weight Set Extraction
The second phase identifies which BNN weights ($w$) produce control outputs within the safe range established in Phase 1. This is achieved through an incremental algorithm that samples weights and employs Lipschitz constant estimation to verify neighborhoods of weights. By accumulating these verified neighborhoods, the framework builds a "Safe Weight Set" ($W$) that is not restricted to simple geometric shapes, allowing for more precise and larger safe regions.
Experimental Results
The framework was evaluated on both linear benchmarks (Unstable LDS, Pendulum, and Collision Avoidance) and complex nonlinear tasks (Dubin’s Car, 2D Oscillator, Academic 3D, Bicycle Steering, Chesi 3, and the 7D LALO20 system).
Key Findings
- Superior Probability Measures: On the "Unstable LDS" benchmark, the proposed method achieved a safe weight set probability measure of 90%, whereas previous state-of-the-art methods yielded measures as low as $3.6 \times 10^{-41}$.
- Increased Safe Radii: The verified radius of safe weight sets was consistently 2x to 4x larger than those produced by existing methods.
- Scalability to High Dimensions: The method successfully verified the 7-dimensional LALO20 nonlinear system with a safe weight set probability measure of 95%.
- Efficiency: Polynomial optimization proved more straightforward and efficient for nonlinear systems compared to MILP or SMT-solver-based inductive invariant checking.
Conclusion and Future Work
By using a "Safe Control Set" as an intermediary, this method successfully maps BNN weights to control inputs and subsequently to system safety. This avoids the computational explosion and errors associated with system discretization. While the method shows significant promise, future work will address the sensitivity of the algorithm to state-space partitioning and explore extensions to stochastic systems governed by practical real-world noise.
Summary updated