Unsupervised Traffic Accident Detection in First-Person Videos

Yu Yao, Mingze Xu, Yuchen Wang, David J. Crandall, Ella M. Atkins

TL;DR

This paper introduces an unsupervised method for detecting traffic accidents in first-person driving videos by predicting the future locations of vehicles and other road participants. By monitoring deviations between predicted and actual trajectories, the system can identify anomalies without needing labeled accident data, outperforming existing state-of-the-art methods.

What Problem Does This Paper Address?

Autonomous driving systems and advanced driver assistance systems need to reliably detect and react to abnormal events on the road, such as traffic violations and accidents. However, existing video anomaly detection methods face two major challenges: they often assume static camera viewpoints (unsuitable for dashboard cameras), and they rely on extensive, hand-labeled datasets that struggle to cover the vast, long-tailed distribution of rare, unexpected events. This makes it difficult to train models that can generalize to novel accident scenarios.

Methods

The core of the proposed approach is an unsupervised Future Object Localization (FOL) model. This model takes as input current object bounding boxes, image features, and ego-vehicle motion information. It then predicts the future bounding box trajectories of traffic participants for several frames ahead. The key novelty lies in how anomalies are detected: instead of trying to predict entire video frames (which is difficult with moving cameras), the system focuses on the accuracy and consistency of these predicted object trajectories.

Three anomaly detection strategies are proposed:

  1. Predicted Bounding Boxes - Accuracy: This method measures the overlap (Intersection over Union - IoU) between the predicted bounding boxes and the actual observed bounding boxes. A low IoU suggests a deviation from expected motion.
  2. Predicted Box Mask - Accuracy: To handle potential tracking inaccuracies, this strategy converts bounding boxes into binary masks and calculates IoU between predicted and observed masks. This is less sensitive to precise tracking errors.
  3. Predicted Bounding Boxes - Consistency: This approach leverages multiple previous FOL predictions. It calculates the standard deviation (STD) of predicted bounding box components (center coordinates, width, height) over several future frames. High STD indicates inconsistent predictions, suggesting an abnormal event.

The system also incorporates ego-motion prediction to account for the camera's movement and a mechanism to handle temporarily missed objects by using their last predicted location.

Datasets

  • Training: The model is trained on the Honda Egocentric View Intersection (HEV-I) dataset, which consists of normal driving videos from dashboard cameras at intersections.
  • Evaluation: The approach is evaluated on:
  • AnAn Accident Detection (A3D): A new dataset introduced by the authors, comprising approximately 1,500 video clips of diverse real-world traffic accidents collected from YouTube dashboard cameras. It includes various weather conditions, locations, and participant types, with labels for anomaly start/end times and ego-vehicle involvement.
  • Street Accidents (SA) [2]: A publicly available dataset of on-road accidents from Taiwan, also collected via dashboard cameras.

Key Insights and Ideas

  • Predicting trajectories over frames is more robust than predicting full frames: For moving cameras, predicting future object locations is more feasible and directly relevant to detecting motion anomalies than trying to reconstruct entire video frames.
  • Unsupervised detection is crucial for rare events: The long-tailed nature of traffic accidents means that collecting comprehensive labeled data for all possible scenarios is impractical. An unsupervised approach that learns "normal" behavior and flags deviations is essential.
  • Consistency of predictions is a strong anomaly indicator: Even if object detection or tracking is imperfect during an accident, the inconsistency of future predictions over time can reliably signal an abnormal event.
  • Ego-motion is a critical factor: Accurately predicting the ego-vehicle's own movement is necessary to distinguish between anomalies caused by other road users and those caused by the camera's motion.

Key Results

  • The proposed unsupervised method significantly outperforms existing state-of-the-art methods on both the A3D and SA datasets, as measured by AUC (Area Under the Curve) for frame-level anomaly detection.
  • The "FOL-MaxSTD" strategy (using the maximum STD of predicted bounding boxes) achieved the best performance, demonstrating the effectiveness of the consistency-based approach.
  • The FOL model itself showed reasonable prediction accuracy (FDE, ADE, FIOU) on accident datasets, though slightly worse than on normal driving data, which is expected given the presence of accidents.
  • Qualitative results show the system successfully identifying accidents, with anomaly scores rising sharply when events occur and recovering as the scene stabilizes.

Limitations and Future Work

  • False Alarms/Negatives: The paper acknowledges that the method can produce false alarms if an object's prediction is consistently inconsistent due to factors like occlusion by trees (as shown in Figure 6). False negatives can occur if participants are completely occluded or their motion appears normal from the ego-vehicle's perspective.
  • Focus on all objects equally: The current model treats all detected objects equally, which might not be optimal. Future work could involve prioritizing more critical objects or regions of interest.
  • Difficulty with low-resolution/occluded objects: The paper notes that predicting future locations can be difficult for low-resolution objects, heavily occluded objects, or in very dense traffic.
  • Future directions: The authors suggest exploring methods that can better handle these challenging scenarios and potentially incorporate more contextual information.

Summary updated

Abstract

Recognizing abnormal events such as traffic violations and accidents in natural driving scenes is essential for successful autonomous driving and advanced driver assistance systems. However, most work on video anomaly detection suffers from two crucial drawbacks. First, they assume cameras are fixed and videos have static backgrounds, which is reasonable for surveillance applications but not for vehicle-mounted cameras. Second, they pose the problem as one-class classification, relying on arduously hand-labeled training datasets that limit recognition to anomaly categories that have been explicitly trained. This paper proposes an unsupervised approach for traffic accident detection in first-person (dashboard-mounted camera) videos. Our major novelty is to detect anomalies by predicting the future locations of traffic participants and then monitoring the prediction accuracy and consistency metrics with three different strategies. We evaluate our approach using a new dataset of diverse traffic accidents, AnAn Accident Detection (A3D), as well as another publicly-available dataset. Experimental results show that our approach outperforms the state-of-the-art. Code and the dataset developed in this work are available at: https:llgithub.comlMoonBtvdltad-IROS2019.
BibTeX
@inproceedings{yao2019unsupervised,
  author = {Yu Yao and Mingze Xu and Yuchen Wang and David J. Crandall and Ella M. Atkins},
  title = {{Unsupervised Traffic Accident Detection in First-Person Videos}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=StTEopXg5Bu},
}
Figure