Abstract
This paper introduces the Near-Miss Incident Database (NIDB), a large-scale collection of over 4,500 traffic "close calls" captured by taxi dashcams. To address the data scarcity of hazardous driving events in existing datasets, the authors propose a "semantic flow" approach that isolates potential hazards from background noise. Their AI system achieves near-human accuracy in recognizing and detecting near-miss incidents, offering a significant step forward for autonomous vehicle safety and advanced driver assistance systems (ADAS).
Introduction
While self-driving cars and ADAS are becoming more common, they suffer from a significant "data scarcity" problem regarding hazardous scenarios. Most driving datasets, such as KITTI or Caltech Pedestrian, consist of thousands of hours of normal, safe driving. Consequently, AI models have plenty of examples of routine driving but very few examples of critical events, such as narrowly avoiding a collision with a cyclist.
The authors argue that for a vehicle to drive safely, it must understand the nuance of a near-miss—situations where a collision was avoided only through urgent braking or steering. Detecting these events is difficult because the visual difference between a routine action (e.g., a person crossing the street) and a near-miss is subtle and often obscured by "noisy" backgrounds in complex urban environments.
Methodology
The researchers built upon an existing state-of-the-art motion descriptor called Trajectory-pooled Deep-convolutional Descriptors (TDD). They introduced two major technical improvements to adapt this for traffic safety:
- NIDB Pre-training: Instead of relying solely on general action datasets (like sports videos), they performed a two-step fine-tuning process using their specific traffic incident data. This domain adaptation helps the neural network focus on vehicle-specific motions.
- Semantic Flow: Standard motion analysis looks at the whole frame, which includes distracting elements like moving trees or shadows. The authors used SegNet to identify specific objects—bicycles, pedestrians, and vehicles—and separated "foreground" motion (the potential hazard) from the "background." By filtering trajectories this way, the model focuses its attention on the actual source of danger.
Data Resources
The cornerstone of this research is the Near-Miss Incident Database (NIDB). This extensive dataset contains over 6,200 videos (approximately 1.3 million frames) collected from more than 100 taxis between 2006 and 2015. These videos were automatically recorded when a vehicle experienced a sudden deceleration exceeding 0.5 G, capturing critical moments of potential danger across both day and night conditions.
The dataset is categorized into six "near-miss" classes based on the type of object involved—bicycle, pedestrian, or vehicle—and the associated risk level. Risk is classified as "High" (hi) or "Low" (lw) based on a Time-to-Collision (TTC) threshold of 0.5 seconds. A large set of background (bg) videos representing normal driving conditions is also included.
Performance and Insights
The authors evaluated their model on two tasks: Recognition (identifying the type of incident) and Detection (identifying the incident within a stream of background footage).
- Near-Miss Recognition: The best model achieved 64.5% accuracy, trailing the human baseline (68.4%) by only a few percentage points. This human baseline is surprisingly low, highlighting the inherent ambiguity in judging near-misses from a single camera view.
- Near-Miss Detection: In the more difficult task of picking out incidents from background noise, the model achieved 61.3% accuracy.
- Impact of Semantic Flow: By explicitly telling the model to focus on pedestrians or vehicles while ignoring buildings (via Semantic Flow), performance in temporal detection improved by 13.9% compared to standard TDD methods.
Future Directions
While the results are promising, several limitations remain. The system currently uses a single camera; incorporating depth data from LiDAR or Stereo cameras would likely improve Time-to-Collision estimations. Furthermore, while the current model detects a near-miss as it happens, the authors suggest that the next logical step is accident anticipation—predicting that a near-miss is about to occur seconds before the driver takes evasive action.
Summary updated