Drive Video Analysis for the Detection of Traffic Near-Miss Incidents

Hirokatsu Kataoka, Teppei Suzuki, Shoko Oikawa, Yasuhiro Matsui, Yutaka Satoh

Abstract

This paper introduces the Near-Miss Incident Database (NIDB), a large-scale collection of over 4,500 traffic "close calls" captured by taxi dashcams. To address the data scarcity of hazardous driving events in existing datasets, the authors propose a "semantic flow" approach that isolates potential hazards from background noise. Their AI system achieves near-human accuracy in recognizing and detecting near-miss incidents, offering a significant step forward for autonomous vehicle safety and advanced driver assistance systems (ADAS).

Introduction

While self-driving cars and ADAS are becoming more common, they suffer from a significant "data scarcity" problem regarding hazardous scenarios. Most driving datasets, such as KITTI or Caltech Pedestrian, consist of thousands of hours of normal, safe driving. Consequently, AI models have plenty of examples of routine driving but very few examples of critical events, such as narrowly avoiding a collision with a cyclist.

The authors argue that for a vehicle to drive safely, it must understand the nuance of a near-miss—situations where a collision was avoided only through urgent braking or steering. Detecting these events is difficult because the visual difference between a routine action (e.g., a person crossing the street) and a near-miss is subtle and often obscured by "noisy" backgrounds in complex urban environments.

Methodology

The researchers built upon an existing state-of-the-art motion descriptor called Trajectory-pooled Deep-convolutional Descriptors (TDD). They introduced two major technical improvements to adapt this for traffic safety:

  1. NIDB Pre-training: Instead of relying solely on general action datasets (like sports videos), they performed a two-step fine-tuning process using their specific traffic incident data. This domain adaptation helps the neural network focus on vehicle-specific motions.
  2. Semantic Flow: Standard motion analysis looks at the whole frame, which includes distracting elements like moving trees or shadows. The authors used SegNet to identify specific objects—bicycles, pedestrians, and vehicles—and separated "foreground" motion (the potential hazard) from the "background." By filtering trajectories this way, the model focuses its attention on the actual source of danger.

Data Resources

The cornerstone of this research is the Near-Miss Incident Database (NIDB). This extensive dataset contains over 6,200 videos (approximately 1.3 million frames) collected from more than 100 taxis between 2006 and 2015. These videos were automatically recorded when a vehicle experienced a sudden deceleration exceeding 0.5 G, capturing critical moments of potential danger across both day and night conditions.

The dataset is categorized into six "near-miss" classes based on the type of object involved—bicycle, pedestrian, or vehicle—and the associated risk level. Risk is classified as "High" (hi) or "Low" (lw) based on a Time-to-Collision (TTC) threshold of 0.5 seconds. A large set of background (bg) videos representing normal driving conditions is also included.

Breakdown of video samples across near-miss categories and background

Performance and Insights

The authors evaluated their model on two tasks: Recognition (identifying the type of incident) and Detection (identifying the incident within a stream of background footage).

  • Near-Miss Recognition: The best model achieved 64.5% accuracy, trailing the human baseline (68.4%) by only a few percentage points. This human baseline is surprisingly low, highlighting the inherent ambiguity in judging near-misses from a single camera view.
  • Near-Miss Detection: In the more difficult task of picking out incidents from background noise, the model achieved 61.3% accuracy.
  • Impact of Semantic Flow: By explicitly telling the model to focus on pedestrians or vehicles while ignoring buildings (via Semantic Flow), performance in temporal detection improved by 13.9% compared to standard TDD methods.

Future Directions

While the results are promising, several limitations remain. The system currently uses a single camera; incorporating depth data from LiDAR or Stereo cameras would likely improve Time-to-Collision estimations. Furthermore, while the current model detects a near-miss as it happens, the authors suggest that the next logical step is accident anticipation—predicting that a near-miss is about to occur seconds before the driver takes evasive action.

Summary updated

Abstract

Because of their recent introduction, self-driving cars and advanced driver assistance system (ADAS) equipped vehicles have had little opportunity to learn, the dangerous traffic (including near-miss incident) scenarios that provide normal drivers with strong motivation to drive safely. Accordingly, as a means of providing learning depth, this paper presents a novel traffic database that contains information on a large number of traffic near-miss incidents that were obtained by mounting driving recorders in more than 100 taxis over the course of a decade. The study makes the following two main contributions: (i) In order to assist automated systems in detecting near-miss incidents based on database instances, we created a large-scale traffic near-miss incident database (NIDB) that consists of video clip of dangerous events captured by monocular driving recorders. (ii) To illustrate the applicability of NIDB traffic near-miss incidents, we provide two primary database-related improvements: parameter fine-tuning using various near-miss scenes from NIDB, and foreground/background separation into motion representation. Then, using our new database in conjunction with a monocular driving recorder, we developed a near-miss recognition method that provides automated systems with a performance level that is comparable to a human-level understanding of near-miss incidents (64.5% vs. 68.4% at near-miss recognition, 61.3% vs. 78.7% at near-miss detection).
BibTeX
@inproceedings{kataoka2018drive,
  author = {Hirokatsu Kataoka and Teppei Suzuki and Shoko Oikawa and Yasuhiro Matsui and Yutaka Satoh},
  title = {{Drive Video Analysis for the Detection of Traffic Near-Miss Incidents}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2018},
  doi = {10.1109/icra.2018.8460812},
}
Figure