TL;DR
This paper introduces DoTA, the first large-scale dataset for detecting traffic anomalies in driving videos, encompassing "when," "where," and "what" aspects. They also propose STAUC, a novel evaluation metric that better captures spatial localization performance than traditional AUC, addressing a key limitation in current video anomaly detection research.
What Problem Does This Paper Address?
Accurate perception of accident scenarios is crucial for advanced driver assistance systems (ADAS). This involves understanding when an anomaly occurs, where it happens in the scene, and what type of anomaly it is. While video anomaly detection (VAD) has been studied, existing datasets for egocentric traffic videos are often small and lack comprehensive annotations for all three aspects. This makes it difficult to develop and evaluate models that can truly understand and react to traffic anomalies in real-world driving scenarios. The paper aims to bridge this gap by providing a large-scale dataset and a more robust evaluation metric.
Methods
The authors propose a "when-where-what" pipeline for traffic anomaly detection. This pipeline is mapped to two computer vision tasks: Video Anomaly Detection (VAD) for identifying the temporal occurrence and spatial location of anomalies, and Video Action Recognition (VAR) for classifying the type of anomaly.
For VAD, they benchmarked several unsupervised methods, including:
- Frame-level methods: ConvAE, ConvLSTMAE (autoencoders that reconstruct input frames), and AnoPred (predicting future frames).
- Object-centric methods: TAD (predicting object trajectories) and a proposed TAD+ML variant incorporating margin learning.
- An Ensemble method combining appearance-based (AnoPred+Mask) and motion-based (TAD+ML) approaches.
For VAR, they benchmarked established methods like R(2+1)D and SlowFast.
A key methodological contribution is the Spatial-Temporal Area Under Curve (STAUC) metric. Unlike traditional AUC, which uses frame-level scores, STAUC considers the spatial overlap between predicted anomaly scores and ground truth anomalous regions, providing a more nuanced evaluation of localization capabilities.
Datasets
The paper introduces the Detection of Traffic Anomaly (DoTA) dataset.
- Size: Contains 4,677 driving videos.
- Annotations: Provides temporal (start/end times), spatial (object bounding boxes), and categorical (18 anomaly types) annotations.
- Distinction: It is claimed to be the largest traffic anomaly dataset to date and the first to support "when-where-what" analysis.
Existing datasets mentioned for comparison include StreetAccident, A3D, and DADA.
Key Insights and Ideas
- AUC is insufficient for VAD: Traditional AUC metrics can be misleading because they average anomaly scores across frames, masking poor spatial localization. Methods with similar AUCs can have vastly different abilities to pinpoint anomalies.
- STAUC offers better evaluation: The proposed STAUC metric, by incorporating spatial overlap, provides a more accurate assessment of VAD performance, especially in distinguishing methods that can localize anomalies effectively.
- Motion and object location are key: VAD methods that focus on object motion and location (like TAD) tend to outperform appearance-based methods, highlighting the importance of understanding dynamic scene elements in traffic anomaly detection.
- Ensembling helps: Combining appearance and motion-based VAD methods through late fusion (average pooling of scores) yields the best overall performance, suggesting that a multi-modal approach is beneficial.
- Traffic anomaly classification is hard: Even state-of-the-art VAR models struggle with classifying traffic anomalies, especially for non-ego events or when anomalies are visually similar or occluded.
Key Results
- DoTA Dataset: The DoTA dataset is presented as a significant resource for VAD and VAR research, offering rich annotations.
- VAD Benchmarking: The Ensemble method achieved the highest AUC (73.0) and STAUC (48.5) among unsupervised VAD methods. TAD+ML also performed strongly, outperforming AnoPred.
- STAUC vs. AUC: The paper demonstrates that methods with high AUC can have significantly lower STAUC, indicating poor localization. For instance, AnoPred achieved 67.5 AUC but only 24.4 STAUC, while AnoPred+Mask had lower AUC (64.8) but much higher STAUC (42.1).
- VAR Benchmarking: State-of-the-art VAR methods like TRN achieved the highest mAP (15.3) in online action detection, but overall performance was still low, indicating the difficulty of the task.
- Per-class Analysis: STAUC revealed significant differences in localization performance across anomaly types, offering insights into which categories are more challenging.
Limitations and Future Work
- Subjectivity of Annotations: The authors acknowledge that labeling traffic anomalies, especially start and end times, can be subjective. They mitigated this by using multiple annotators and merging their judgments.
- VAR Performance: Current VAR methods struggle with traffic anomaly classification, particularly for non-ego anomalies and visually similar events.
- Future Work: The authors suggest exploring spatial modeling to further boost supervised VAD performance. They also propose that DoTA can be used for other tasks like object detection, visual saliency, and validation of autonomous driving systems. Further research into combining frame-level and object-centric VAD methods is also encouraged.
Summary updated