Abstract
There is an urgent societal need to assess whether autonomous vehicles (AVs) are safe enough. From published quantitative safety and reliability assessments of AVs, we know that, given the goal of predicting very low rates of accidents, road testing alone requires infeasible numbers of miles to be driven. However, previous analyses do not consider any knowledge prior to road testing - knowledge which could bring substantial advantages if the AV design allows strong expectations of safety before road testing. We present the advantages of a new variant of Conservative Bayesian Inference (CBI), which uses prior knowledge while avoiding optimistic biases. We then study the trend of disengagements (take-overs by human drivers) by applying Software Reliability Growth Models (SRGMs) to data from Waymo's public road testing over 51 months, in view of the practice of software updates during this testing. Our approach is to not trust any specific SRGM, but to assess forecast accuracy and then improve forecasts. We show that, coupled with accuracy assessment and recalibration techniques, SRGMs could be a valuable test planning aid.
Introduction: The "RAND Challenge"
Proving that an autonomous vehicle (AV) is safer than a human driver is a monumental statistical challenge. To demonstrate with 95% confidence that an AV is 20% safer than a human driver, a fleet would need to drive approximately 5 billion miles without a single fatality. At current testing rates, this "RAND challenge" would take centuries to complete.
This paper identifies two critical flaws in standard safety assessments:
- Ignoring Prior Knowledge: Classical statistics often ignore valuable engineering data from simulations, formal verification, and safety audits.
- The Static Assumption: Most models assume AV reliability is constant, ignoring the reality that software is iteratively updated to fix bugs and improve performance.
Methodology
The researchers propose two distinct statistical frameworks to address these gaps:
Conservative Bayesian Inference (CBI)
Unlike standard Bayesian methods that require a complete (and often arbitrary) prior probability distribution, CBI utilizes partial prior knowledge. For instance, an assessor might state an 80% confidence that the failure rate is below a certain threshold. CBI then identifies the most "pessimistic" distribution consistent with this claim, ensuring that safety assessments remain conservative and avoid over-optimistic biases.
Software Reliability Growth Models (SRGMs) and Recalibration
SRGMs are used to track and predict reliability improvements based on the history of "disengagements" (human take-overs). Because no single model is universally perfect, the authors employ a recalibration technique. This involves evaluating a model's past predictive performance and adjusting future forecasts accordingly to improve accuracy.
The methods were validated using 51 months of Waymo public road testing data (comprising 528 disengagements) and RAND study benchmarks for fatality-free mile requirements.
Key Insights and Ideas
A primary finding is that prior knowledge acts as a force multiplier. Strong evidence from non-road testing (e.g., a verified safety "watchdog" system) can drastically reduce the required road-testing mileage.
The paper also introduces the concept of "Fatality Compensation" math. This provides a rigorous way to calculate the additional miles a company must drive to "make up" for an observed fatality while maintaining their safety claim. Interestingly, the authors proved a mathematical "ceiling" on these extra miles, regardless of the confidence level.
Furthermore, the authors emphasize that disengagements are not a direct proxy for fatalities. While useful for tracking software maturity, a low disengagement rate does not guarantee protection against rare, catastrophic "edge case" failures.
Key Results
The application of CBI demonstrates significant efficiency gains. To claim an AV is as safe as a human ($1.09 \times 10^{-8}$ fatalities per mile) with 95% confidence:
- The Classical (RAND) approach requires 275 million fatality-free miles.
- CBI with a strong prior (90% confidence) requires only 69 million fatality-free miles.
Analysis of Waymo's data showed a clear reliability growth trend. By the end of the 51-month study, the "Median Miles to next Disengagement" (MMTD) was estimated at approximately 7,000–8,000 miles. Recalibration proved essential, as it significantly reduced the wild disagreements between different SRGMs, making them viable for test planning.
Conclusion and Future Work
While CBI offers a path forward, justifying initial "prior beliefs" remains a challenge in safety engineering. Future work should address stratified testing—accounting for the fact that miles driven in complex urban environments are more informative than highway miles—and the human factors involved in safety driver interventions.
Summary updated