Validating the safety of autonomous vehicles (AVs) is a "needle in a haystack" problem because failures are rare. This paper improves Adaptive Stress Testing (AST) by training a Bayesian Neural Network to recognize "critical states" based on human expert labels. By incorporating these human-identified dangers into the search process, the system uncovers more diverse, aggressive, and meaningful failure scenarios than traditional automated methods.
The Adaptive Stress Testing Framework
To trust autonomous vehicles, we need to test them against "edge cases"—rare, dangerous scenarios where the system might fail. A popular method for this is Adaptive Stress Testing (AST), which uses Reinforcement Learning (RL) to "stress" the vehicle in simulation to find failures.
However, standard AST usually relies on a mathematical probability model of the environment. This works fine for continuous variables (like sensor noise), but it struggles with discrete high-level decisions (like whether a surrounding car should change lanes or brake). In these discrete settings, it’s hard to define what a "likely" failure looks like without hand-crafting complex heuristics. If the heuristic is too simple, the search misses complex failures; if it's too rigid, it limits the variety of scenarios discovered.
Methods: Human-based Critical State (HCS) Classification
The authors propose a data-driven approach to guide the failure search using Human-based Critical State (HCS) Classification. They use Monte Carlo Tree Search (MCTS) as the "stress tester" to control the environment agents surrounding the autonomous vehicle (the System Under Test, or SUT).
The Bayesian Classifier
The core innovation is the reward function. Instead of just rewarding a crash, the system rewards the tester for pushing the AV into "critical states" identified by a Bayesian Neural Network (BNN). This classifier was trained on driving trajectories where humans labeled states as either "dangerous" or "safe."
Because it's a Bayesian network, the classifier provides both a prediction (is this dangerous?) and a confidence score (variance). If the BNN is highly confident, the reward is based on the "danger" score. If the BNN is uncertain, the system automatically falls back to a simple distance-based heuristic. This prevents the RL solver from being misled by the classifier's "blind spots."
Key Results
The experiments were conducted in the highway-env OpenAI Gym environment. To objectively measure "danger" without relying on the classifier itself, the authors used Responsibility-Sensitive Safety (RSS)—a mathematical model developed by Mobileye to define safe driving boundaries.
- Increased Severity: Using the HCS reward led to a "right shift" in the distribution of safety violations. In one test configuration, the median proportion of improper responses from the AV increased by ~50% (from 21% to 32% of the trajectory) compared to the heuristic baseline.
- Qualitative Diversity: While simple heuristics often just found cases where the AV tailgated, the HCS method uncovered more complex "cut-off" scenarios where surrounding vehicles forced the AV into sudden, unsafe maneuvers.
- Sample Efficiency: The classifier's performance began to converge with as few as 400 to 800 human-labeled examples, suggesting that this "human-in-the-loop" approach is practically feasible for researchers.
Conclusion and Future Work
By encoding human intuition into a neural network, AST can explore the "pre-failure" space much more effectively than using the agent's internal uncertainty alone. Future work could explore using raw images or bird's-eye-view maps to better capture spatial relationships, as well as active learning to identify which specific states would be most beneficial for a human to label.
Summary updated