Abstract
Designing or learning an autonomous driving policy is undoubtedly a challenging task as the policy has to maintain its safety in all corner cases. In order to secure safety in autonomous driving, the ability to detect hazardous situations, which can be seen as an out-of-distribution (OOD) detection problem, becomes crucial. However, most conventional datasets only provide expert driving demonstrations, although some non-expert or uncommon driving behavior data are needed to implement a safety guaranteed autonomous driving platform. To this end, we present a novel dataset called the R3 Driving Dataset, composed of driving data with different qualities. The dataset categorizes abnormal driving behaviors into eight categories and 369 different detailed situations. The situations include dangerous lane changes and near-collision situations. To further enlighten how these abnormal driving behaviors can be detected, we utilize different uncertainty estimation and anomaly detection methods to the proposed dataset. From the results of the proposed experiment, it can be inferred that by using both uncertainty estimation and anomaly detection, most of the abnormal cases in the proposed dataset can be discriminated.
Introduction
Most autonomous driving (AD) datasets focus on expert driving demonstrations, providing a library of "perfect" behaviors. However, real-world driving is fraught with hazards, such as sudden swerves, lane-weaving, and near-collisions. To build truly defensive autonomous systems, models must not only mimic experts but also recognize when a situation has become dangerous or "out-of-distribution" (OOD). This paper introduces the $R^3$ Driving Dataset, a novel benchmark designed to help AI detect hazardous driving scenarios by providing demonstrations of mixed qualities.
The $R^3$ Driving Dataset
The $R^3$ Driving Dataset was collected using a Hyundai Ioniq test vehicle equipped with a comprehensive sensor suite, including three Velodyne LiDARs, a Mobileye vision sensor, and a high-precision GPS/IMU system.
The dataset is divided into two primary components:
- Expert Data: 42,400 frames of standard driving collected from highways, urban roads, and a dedicated test site (FMTC) in Korea.
- Abnormal Data: 20,355 frames across 369 detailed scenarios based on the Common Accident Data Set (CADaS). These include unstable driving, dangerous overtaking, and near-collisions on both straight roads and crossroads. To ensure safety, these maneuvers were performed in controlled environments.
Methodology: Probing Anomalies
The researchers explored two primary methods for detecting these hazardous OOD situations:
- Uncertainty Estimation via Mixture Density Networks (MDN): By predicting a probability distribution of actions, the model can calculate Epistemic Uncertainty (the model’s lack of knowledge about a situation) and Pi-entropy (the complexity of the action distribution).
- Reconstruction Loss via Variational Autoencoders (VAE): A VAE learns to compress and reconstruct safe driving data. When it encounters a hazardous situation it hasn't seen before, the reconstruction error increases significantly.
Results and Analysis
The performance of these methods was evaluated using the Area Under the Receiver Operating Characteristic curve (AUROC). The findings revealed a clear distinction between "internal" and "external" hazards.
As shown in the performance curves, Epistemic Uncertainty serves as an effective "internal alarm," excelling at detecting when the ego-vehicle itself is behaving abnormally (e.g., lane-weaving). Conversely, Reconstruction Loss acts as an "external alarm," proving significantly better at identifying hazards caused by other vehicles, such as dangerous cut-ins or near-collisions.
Furthermore, the study found that temporal context matters. Utilizing a sequence of five frames improved uncertainty-based detection (reaching an AUROC of 0.901 for crossroad anomalies) but slightly degraded reconstruction-based performance. This suggests that behavioral anomalies are best identified over time, while spatial anomalies are best caught in a single snapshot.
Conclusion
The $R^3$ Driving Dataset provides a critical benchmark for developing defensive autonomous driving policies. By demonstrating that uncertainty estimation and reconstruction loss are complementary—one catching internal errors and the other catching external threats—this work paves the way for more robust, safety-aware autonomous systems. Future work will focus on integrating these detection mechanisms into real-time defensive control policies.
Summary updated