Does Redundancy in AI Perception Systems Help to Test for Super-Human Automated Driving Performance?

Hanno Gottschalk, Matthias Rottmann, Maida SaltagicarXiv 2021 · arXiv

Abstract

Proving that automated driving systems (ADS) are safer than human drivers requires an astronomical amount of testing data, leading to prohibitive economic costs. While redundancy in AI subsystems is often proposed to mitigate this testing burden by assuming independent failure modes, this study demonstrates that neural network errors are highly correlated in practice. Through statistical modeling and empirical evaluation on datasets like EMNIST and CIFAR10, we show that the "redundancy trick" fails because models tend to fail on the same difficult examples. Furthermore, proving the independence required for data reduction is nearly as costly as the original safety test, suggesting that redundancy alone cannot solve the validation crisis for autonomous vehicles.

Introduction

The statistical validation of safety remains the primary hurdle for the deployment of autonomous vehicles (AVs). To demonstrate that an AV is "super-human"—for instance, 10 to 100 times safer than a human driver—it must be tested over billions of kilometers to provide statistically significant evidence of a lower failure rate. In Germany, where a fatal accident occurs roughly every 250 million kilometers, the data requirements for such a proof are staggering.

The Economic Infeasibility of Brute-Force Testing

The sheer volume of data required for brute-force safety validation leads to an economic paradox. The costs associated with data acquisition and, more critically, manual labeling of exabytes of sensor data are immense.

Table 1: Estimated costs and data requirements for proving super-human performance in autonomous driving.

As shown in Table 1, the estimated total cost for labeling the necessary frames ranges from approximately 1.16 trillion to over 518 quadrillion Euros. Given that the lower bound already approaches the scale of a national GDP, traditional statistical testing is economically unviable.

Methods

To investigate whether redundancy can alleviate this burden, the authors employed a multi-faceted approach:

  1. Statistical Modeling: Deriving the "leverage factor" for data reduction under varying assumptions of subsystem independence.
  2. Correlation Analysis: Measuring Pearson correlation coefficients of error indicator variables across diverse neural network architectures and initializations.
  3. Decorrelation Training: Implementing a novel loss function to penalize joint errors and force networks to fail on different data types.
  4. Multi-Sensor Simulation: Using 3D-MNIST data projected from multiple angles to simulate diverse physical sensor inputs (e.g., LiDAR vs. Camera).

Key Insights and Results

The study reveals several critical challenges to the redundancy hypothesis:

  • The Independence Myth: Despite variations in architecture and training data, neural networks exhibit high error correlation (Pearson coefficients ≈ 0.57 to 0.74). They consistently struggle with the same "hard" examples.
  • The Accuracy-Independence Trade-off: While it is possible to force correlation down to zero using specialized loss functions, this typically results in a significant drop in individual model accuracy (e.g., from 91% to 72%). A system of independent but mediocre models often provides less overall safety than a single high-performing model.
  • The Cost of Proving Independence: Utilizing the redundancy shortcut requires prior proof that subsystems are truly independent. The authors demonstrate that the data required to prove this independence is nearly equivalent to the data required for the original brute-force safety test.
  • Sensors vs. Architectures: Using different 2D projections of 3D-MNIST (simulating multi-sensor setups) reduced mean correlation to 0.25. While more effective than architectural changes, this still only provided a moderate reduction in testing requirements—roughly one order of magnitude—which is insufficient to make brute-force testing viable.

Conclusion

Redundancy is not a "silver bullet" for the statistical validation of autonomous driving safety. The high correlation of errors in AI perception systems prevents the massive data reductions needed to make brute-force testing economically viable. Future work must explore alternative strategies, such as high-fidelity synthetic data generation in simulators, formal mathematical verification of model behavior, and targeted testing on rare "corner case" scenarios rather than accumulated highway miles.

Summary updated

Abstract

While automated driving is often advertised with better-than-human driving performance, this work reviews that it is nearly impossible to provide direct statistical evidence on the system level that this is actually the case. The amount of labeled data needed would exceed dimensions of present day technical and economical capabilities. A commonly used strategy therefore is the use of redundancy along with the proof of sufficient subsystems' performances. As it is known, this strategy is efficient especially for the case of subsystems operating independently, i.e. the occurrence of errors is independent in a statistical sense. Here, we give some first considerations and experimental evidence that this strategy is not a free ride as the errors of neural networks fulfilling the same computer vision task, at least for some cases, show correlated occurrences of errors. This remains true, if training data, architecture, and training are kept separate or independence is trained using special loss functions. Using data from different sensors (realized by up to five 2D projections of the 3D MNIST data set) in our experiments is more efficiently reducing correlations, however not to an extent that is realizing the potential of reduction of testing data that can be obtained for redundant and statistically independent subsystems.
BibTeX
@misc{gottschalk2021does,
  author = {Hanno Gottschalk and Matthias Rottmann and Maida Saltagic},
  title = {{Does Redundancy in AI Perception Systems Help to Test for Super-Human Automated Driving Performance?}},
  year = {2021},
  eprint = {2112.04758},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2112.04758},
}
Figure