Road Defect Detection in Street View Images using Texture Descriptors and Contour Maps

David Abou Chacra, Henry A. Leopold, Jeremy Pinto, Norman Lunscher, Georges Younes, John ZelekJournal of Computational Vision and Imaging Systems 2016 · DOI

Abstract

Road quality assessment is a crucial part of municipal infrastructure maintenance, budget management, and upgrade planning. Consistently monitoring road conditions over time is challenging, particularly in large urban areas with vast property footprints. This paper reviews current assessment methods and introduces a novel algorithm designed to identify distressed road regions from Street View images and pinpoint specific cracks. The system predicts distressed regions by computing Fisher vectors on local SIFT descriptors and classifying them using a Support Vector Machine (SVM). This is followed by a comparison to weighted contour maps within identified regions to locate defects and predict crack severity. Results on a manually annotated dataset demonstrate the viability of this cost-effective system for municipal-level road quality assessment.

Introduction

Maintaining municipal infrastructure is a massive financial burden; in the U.S. alone, road investment reaches $91 billion annually, yet often falls short of preventing decline. Traditionally, road quality assessment has relied on two methods: manual surveys, which are slow and subjective, or high-end automated systems like the Laser Crack Measurement System (LCMS), which are prohibitively expensive and produce massive data volumes. This research aims to bridge the gap by leveraging publicly available Street View imagery to automate the detection of cracks and potholes at a fraction of the cost.

Methodology

The proposed system utilizes a multi-stage computer vision pipeline to process standard color images and extract meaningful distress data.

Overview of the multi-stage road defect detection pipeline, from segmentation to crack pinpointing.
  1. Road Segmentation: The algorithm first isolates the road from background elements like sky and buildings using texture-based segmentation.
  2. Texture Descriptors: The system samples windows from the road and extracts SIFT (Scale-Invariant Feature Transform) descriptors, encoded into Fisher Vectors. This treats road quality as a texture problem, distinguishing the "rough" texture of distressed pavement from smooth asphalt.
  3. Classification: An SVM classifies these vectors as "good quality" or "damaged."
  4. Gaussian Voting: A window-by-window voting scheme creates a smooth map of distressed regions.
  5. Crack Pinpointing: In flagged regions, an Ultrametric Contour Map (UCM) identifies exact crack coordinates. The UCM response intensity categorizes crack severity into mild (green), medium (orange), or severe (red).

Key Insights

  • Texture over Geometry: By prioritizing texture descriptors over simple geometric line detection, the system is more robust against visual noise like shadows from power lines or tire marks.
  • Two-Stage Filtering: Identifying distressed regions before pinpointing cracks significantly reduces false positives. Crack-like artifacts are ignored if they appear on a road surface that otherwise exhibits a "good" texture.
  • Passive Data Leveraging: While Street View lacks the resolution of specialized LIDAR rigs, its ubiquity and zero cost make it a powerful tool for scalable municipal monitoring.

Results and Evaluation

The system was evaluated against a manually annotated dataset of Street View images. The results indicate high reliability for a screening tool:

  • Recall: 100% — The system successfully identified every instance of road distress in the test set.
  • F-measure: 92.84% — Demonstrating a strong balance between precision and recall.
  • Precision: 86.64% — While some "good" road was flagged as distressed, the low False Positive Rate (0.02%) ensures the system rarely hallucinates defects.
Final output showing pinpointed cracks and their predicted severity levels.

Conclusion and Future Work

While promising, the system is currently limited by the update frequency of Street View data (typically every two years) and the resolution of the images, which may obscure very fine cracks. Future work will involve testing on larger, more diverse datasets and expanding the scope to other municipal assets like sidewalks, bridges, and tunnels.

Summary updated

Abstract

Road quality assessment is a crucial part in municipalities’ workto maintain their infrastructure, plan upgrades, and manage theirbudgets. Properly maintaining this infrastructure relies heavily onconsistently monitoring its condition and deterioration over time.This can be a challenge, especially in larger towns and cities wherethere is a lot of city property to keep an eye on. We review roadquality assessment methods currently employed, and then describeour novel algorithm aimed at identifying distressed road regionsfrom street view images and pinpointing cracks within them. Wepredict distressed regions by computing Fisher vectors on localSIFT descriptors and classifying them with an SVM trained to distinguishbetween road qualities. We follow this step with a comparisonto a weighed contour map within these distressed regionsto identify exact crack and defect locations, and use the contourweights to predict the crack severity. Promising results are obtainedon our manually annotated dataset, which indicate the viability ofusing this cost-effective system to perform road quality assessmentat a municipal level.
BibTeX
@article{chacra2016road,
  author = {David Abou Chacra and Henry A. Leopold and Jeremy Pinto and Norman Lunscher and Georges Younes and John Zelek},
  title = {{Road Defect Detection in Street View Images using Texture Descriptors and Contour Maps}},
  journal = {Journal of Computational Vision and Imaging Systems},
  year = {2016},
  doi = {10.15353/vsnl.v2i1.94},
}
Figure