Detecting Unsigned Physical Road Incidents from Driver-View Images

Alex Levering, Martin Tomko, Devis Tuia, Kourosh Khoshelham

Introduction

Traffic congestion represents a significant economic burden, costing approximately half a billion pounds annually in England alone. While modern computer vision systems effectively identify "signed" incidents—planned events like roadworks marked by cones and barriers—detecting "unsigned" incidents remains a critical challenge. These unplanned hazards, such as fallen trees, flash floods, or sudden vehicle crashes, occur without prior warning signs, posing immediate risks to drivers and autonomous systems.

Methodology: Taxonomy and Deep Learning

To address the lack of systematic classification for these hazards, researchers developed a novel taxonomy using Formal Concept Analysis (FCA). This hierarchy categorizes incidents based on their origin (Man-made vs. Natural) and their physical manifestation (Object vs. Cover).

Taxonomy of physical road incidents, distinguishing between signed and unsigned hazards.

The detection system utilizes a ResNet-34 convolutional neural network (CNN) fine-tuned on a custom dataset. To ensure the model focuses on relevant features, the team employed Class Activation Mapping (CAM) to verify that the AI's "attention" was directed at the road hazard rather than peripheral elements like the sky or roadside vegetation.

Dataset Development

The study introduced a comprehensive "Incident Dataset" comprising 12,630 positive images across eight distinct classes. These images were harvested via automated web scraping across multiple languages to reduce geographic bias.

Examples of the eight unsigned incident classes: Crash, Collapse, Animal, Treefall, Snow, Flood, Landslide, and Fire.

To provide a robust baseline, the model was also trained on 40,000 "negative" examples of normal roads sourced from the Berkeley Deep Drive (BDD100k), Cityscapes, and Geograph UK datasets.

Key Results and Insights

The best-performing model achieved an overall accuracy of 97.15% and an F1-score of 0.89. While classes like "Fire" and "Negative" (normal road) were identified with near-perfect accuracy (>98%), "Flooded Road" proved the most challenging, often confused with wet but passable road surfaces.

A critical finding was the distinction between "Object" and "Cover" incidents. "Cover" hazards (e.g., snow or floods) are semantically difficult because they share textures with normal road surfaces. Furthermore, geographic stratification tests showed a slight performance dip when testing on unfamiliar regions (e.g., Wales), indicating that road appearance varies enough to require spatially diverse training data.

Limitations and Future Directions

While highly accurate at detection, the current model does not yet assess incident severity—for instance, distinguishing between a shallow puddle and impassable floodwater. Future research will focus on quantifying hazard severity and further refining the dataset to eliminate potential sampling biases inherent in web-harvested imagery.

Summary updated

BibTeX
@article{levering2021detecting,
  author = {Alex Levering and Martin Tomko and Devis Tuia and Kourosh Khoshelham},
  title = {{Detecting Unsigned Physical Road Incidents from Driver-View Images}},
  journal = {IEEE Trans. Intell. Veh.},
  year = {2021},
  doi = {10.1109/tiv.2020.2991963},
}
Figure