Towards Anomaly Detection in Dashcam Videos

Sanjay Haresh, Sateesh Kumar, M. Zeeshan Zia, Quoc-Huy Tran

Detecting accidents in dashcam footage is a "long-tail" problem that hand-crafted rules cannot solve. This research introduces RetroTrucks, a large-scale dataset of truck dashcam videos, and evaluates deep learning approaches that learn "normal" driving behavior to identify anomalies. The findings suggest that while standard reconstruction-based models excel with static cameras, they struggle with the complex, dynamic interactions of real-world traffic.

The Challenge of Moving Cameras

Most current Advanced Driver-Assistance Systems (ADAS) rely on explicit models for specific events, such as lane departures or forward collisions. However, road accidents are incredibly diverse; the "long-tail" of possible hazards makes it impossible to program a rule for every scenario.

While anomaly detection offers a potential solution, most existing work focuses on static surveillance cameras. In a dashcam setting, the camera itself is moving, and anomalies are often subtle interactions between vehicles rather than just strange objects appearing. There is a significant lack of both relevant datasets and specialized algorithms for this moving-camera context.

Unsupervised Learning Methods

The study explores two primary families of unsupervised learning that require only "normal" driving data for training:

  1. One-Class Classification (Deep SVDD): A 3D Convolutional Neural Network (CNN) maps normal video clips into a compact "hypersphere" in feature space. Test samples falling outside this sphere are flagged as anomalies.
  2. Reconstruction-Based (3D Autoencoders): The model learns to compress and then reconstruct video clips. It becomes an expert at reconstructing "normal" driving but fails (producing high error) when encountering unseen anomalous events.

Object Interaction Modeling

Standard CNNs process pixels, but accidents are fundamentally about relationships. To address this, the authors augment the base methods with a Graph Convolutional Network (GCN). A Region Proposal Network (RPN) identifies objects (cars, trucks), and the GCN reasons about their interactions over time, providing a "prior" on normal traffic flow.

Reconstruction-based architectures: (a) standard 3D autoencoder, (b) augmented with GCN for object interaction.

Datasets and Benchmarks

The research utilizes several datasets to contrast static and dynamic environments:

  • RetroTrucks: A new dataset of 474 videos curated from truck-mounted dashcams, featuring collisions, near-misses, and rollovers in various conditions.
  • UCSD Ped1/Ped2 & ShanghaiTech: Standard benchmarks featuring static cameras and "visual" anomalies, such as a cyclist on a walking path.
An example of a "visual" anomaly from a static camera dataset: a cyclist on a pedestrian path.

Key Insights

  • Performance Gap: Methods achieving over 90% AUC-ROC on static surveillance perform significantly worse on dashcam footage. The ego-vehicle's motion makes it difficult to distinguish camera movement from actual traffic anomalies.
  • Context vs. Appearance: In older datasets, an anomaly is often a "novel object." In driving, the objects are normal; it is their behavior (e.g., swerving) that is anomalous.
  • The Reconstruction Paradox: Counter-intuitively, a model that is "too good" at reconstruction may fail to detect anomalies. If the bottleneck is too large, the model reconstructs everything—including accidents—reducing the reconstruction error signal.

Results and Future Directions

On the UCSD Ped2 dataset, reconstruction models achieved 0.954 AUC-ROC, but this dropped to 0.712 on RetroTrucks. Adding GCN-based object interaction consistently improved performance by 1-3%, suggesting that while object relationships are key, more sophisticated modeling is required.

Future work will focus on fusing RGB data with optical flow and developing techniques to ignore non-critical visual artifacts like windshield wipers or camera distortions that currently distract anomaly detection models.

Summary updated

Abstract

Inexpensive sensing and computation, as well as insurance innovations, have made smart dashboard cameras ubiquitous. Increasingly, simple model-driven computer vision algorithms focused on lane departures or safe following distances are finding their way into these devices. Unfortunately, the long-tailed distribution of road hazards means that these hand-crafted pipelines are inadequate for driver safety systems. We propose to apply data-driven anomaly detection ideas from deep learning to dashcam videos, which hold the promise of bridging this gap. Unfortunately, there exists almost no literature applying anomaly understanding to moving cameras, and correspondingly there is also a lack of relevant datasets. To counter this issue, we present a large and diverse dataset of truck dashcam videos, namely RetroTrucks, that includes normal and anomalous driving scenes. We apply: (i) one-class classification loss and (ii) reconstruction-based loss, for anomaly detection on RetroTrucks as well as on existing static-camera datasets. We introduce formulations for modeling object interactions in this context as priors. Our experiments indicate that our dataset is indeed more challenging than standard anomaly detection datasets, and previous anomaly detection methods do not perform well here out-of-the-box. In addition, we share insights into the behavior of these two important families of anomaly detection approaches on dashcam data.
BibTeX
@inproceedings{haresh2020towards,
  author = {Sanjay Haresh and Sateesh Kumar and M. Zeeshan Zia and Quoc-Huy Tran},
  title = {{Towards Anomaly Detection in Dashcam Videos}},
  booktitle = {IV},
  year = {2020},
  url = {https://openreview.net/forum?id=p-riLUFRIv},
}
Figure