Crash report data analysis for creating scenario-wise, spatio-temporal attention guidance to support computer vision-based perception of fatal crash risks

Yu Li, Muhammad Monjurul Karim, Ruwen Qin, Zeyi Sun, Zuhui Wang, Zhaozheng YinAccident 2020 · DOI · arXiv

Abstract

This research presents a data-driven "attention guidance" framework designed to enhance computer vision (CV) systems in autonomous vehicles. By mining five years of U.S. fatal crash data from the Fatality Analysis Reporting System (FARS), the authors developed a model that layers historical risk statistics onto real-time object detection. This allows AI to prioritize objects based on their statistical likelihood of involvement in fatal accidents under specific spatio-temporal conditions, effectively providing the AI with "contextual wisdom" derived from historical patterns.

Introduction

While modern Computer Vision (CV) systems have become remarkably adept at detecting pedestrians and vehicles, they often lack the "common sense" or contextual wisdom that human drivers use to anticipate danger. For example, a CV system might detect a pedestrian on a sidewalk and a car in the opposite lane with equal priority. However, historical data might show that at 2:00 AM on a high-speed rural road, that pedestrian represents a significantly higher fatal crash risk than the passing car.

The core challenge in training AI for these situations is that annotated image data for "near-crash" scenarios is extremely scarce. To bridge this gap, the authors leverage the Fatality Analysis Reporting System (FARS)—a massive repository of historical crash reports—to provide a "spatio-temporal" map of risk that guides where a CV model should focus its computational "attention."

Methodology

The researchers employed a multi-stage data analytics pipeline to transform raw crash reports into actionable AI guidance. This framework moves from data selection to factor identification and finally to the generation of guidance rules.

The multi-stage data analytics pipeline used to generate spatio-temporal attention guidance.
  1. Data Decomposition: The team segmented 162,104 fatal crash cases using variables like work zones and school bus involvement. They focused on the "common scenario"—standard driving environments representing roughly 79% of all fatal crashes.
  2. Clustering: Using K-means and Hierarchical clustering, they grouped 184 spatially-defined road types into six distinct clusters based on their temporal crash patterns (e.g., weekday vs. weekend distributions).
  3. Association Rule Mining: For each cluster, the team used association rules to find statistical links between driving scene features (like "May at 6:00 AM") and crash outcomes (like "Pedestrian involved").
  4. Metrics for "Attention": They used three key metrics—Support (frequency), Confidence (likelihood), and Lift (relative probability)—to determine which rules were strong enough to serve as "guidance."
  5. CV Integration: This guidance was integrated into a CV pipeline using YOLO v3 for object detection and DeepLab v3 for road segmentation, allowing the system to "filter" detected objects based on their calculated risk level.

Data Processing and the "Common Scenario"

The primary dataset used was FARS, encompassing U.S. fatal crash data from 2013 to 2017. To ensure the model focused on generalizable driving conditions rather than niche edge cases, the researchers performed a systematic decomposition of the data.

Data decomposition process identifying the 'common scenario' which accounts for 79.05% of fatal crash cases.

By filtering out rare factors such as school bus involvement (0.32%) or work zones (1.97%), the researchers identified a "common scenario" comprising 128,149 cases (79.05%). This subset provided a robust baseline for identifying the spatio-temporal signatures of typical fatal accidents.

Key Insights and Results

The study demonstrates that risk is not uniform across time and space. By identifying "spatio-temporal signatures," the AI can shift focus dynamically. For instance, the model successfully reduced 184 complex road configurations into 6 manageable clusters with high internal similarity.

In one specific example (Cluster 1), the model identified that in May at hour 0, the confidence that a fatal crash would involve a pedestrian is 53.1%, which is 2.2 times higher (Lift) than the average probability for that cluster. Identifying these signatures allows the AI to shift its focus dynamically, prioritizing pedestrians in night-time scenarios where they are statistically more vulnerable.

Qualitative results showed that the "Attention Guidance" allowed the CV model to correctly identify a "risky pedestrian" in the direct travel path while ignoring irrelevant agents like vehicles in a garage or far-off lanes. This reduces "false alarms" and saves computational resources for genuine threats.

Limitations and Future Work

While the framework is a significant step forward, it faces challenges with data sparsity in very specific or rare scenarios where historical data is insufficient for significant association rules. Furthermore, the authors acknowledge the need for rigorous testing with human drivers—using eye-tracking and EEG—to see if this "guidance" actually improves human reaction times or reduces cognitive load.

Future work will also focus on addressing "temporal decay," as crash patterns evolve with infrastructure and vehicle safety technology changes. Finally, the current implementation is a proof-of-concept; future iterations must ensure the entire pipeline (CV + Rule Querying) can run with minimal latency on edge hardware for real-time application.

Summary updated

BibTeX
@misc{li2020crash,
  author = {Yu Li and Muhammad Monjurul Karim and Ruwen Qin and Zeyi Sun and Zuhui Wang and Zhaozheng Yin},
  title = {{Crash report data analysis for creating scenario-wise, spatio-temporal attention guidance to support computer vision-based perception of fatal crash risks}},
  year = {2020},
  eprint = {2109.02710},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.02710},
}
Figure