Measuring urban quality and change through the detection of physical attributes of decay

Andrea Vallebueno, Yong Suk LeeScientific 2023 · DOI

TL;DR

This paper introduces a novel method for measuring urban quality and change over time using Google Street View (GSV) imagery and object detection. By identifying eight key indicators of urban decay, they create a granular urban quality index that can track neighborhood evolution and inform urban planning.

What Problem Does This Paper Address?

Accurately measuring and tracking the quality of urban environments is crucial for societal well-being and effective urban planning. However, this is a significant challenge due to the sheer volume of on-the-ground data required. While street view imagery offers a promising avenue, its reliability and consistency across different locations and over time have been largely unexplored. This paper aims to bridge this gap by developing a scalable and spatially granular method to quantify urban decay and its changes.

Methods

The core of this research lies in leveraging computer vision, specifically object detection, to analyze GSV imagery. The authors trained a YOLOv5 model to identify eight object classes indicative of urban decay: potholes, graffiti, garbage, tents, barred or broken windows, discolored or dilapidated façades, weeds, and utility markings. They developed a comprehensive urban quality index at the street segment level by aggregating the detections of these features. This index allows for the analysis of spatial disparities and temporal dynamics of urban decay. The study also explores the potential for a generalizable model across different cities and discusses strategies for handling variations in GSV image availability and quality.

Datasets

The study utilized several datasets:

  • San Francisco street images: Manually collected and annotated images from San Francisco.
  • STORM graffiti/tagging dataset: A dataset of graffiti images from Athens, used to augment training data.
  • Google Street View (GSV) imagery: Collected from San Francisco, Oakland, Los Angeles, Mexico City, and South Bend for both training and inference.

The authors created two primary training datasets:

  • "Consolidated": Includes all collected imagery.
  • "Ex-Mexico": Excludes Mexico City GSV images to assess the impact of city-specific training data.

Three test sets were generated from GSV imagery for evaluating model performance in:

  • Mission District, San Francisco
  • Mexico City neighborhoods
  • South Bend neighborhoods

Key Insights and Ideas

  • GSV as a powerful tool for urban analytics: The paper demonstrates that GSV imagery, when combined with object detection, can provide a rich and granular dataset for understanding urban quality and change, overcoming the limitations of traditional data sources.
  • Generalizability of urban decay detection: A single object detection model can be trained to identify urban decay features across diverse cities, though performance can vary significantly.
  • Importance of context and local adaptation: While a general model is feasible, its effectiveness is influenced by the specific urban profile of a location. Fine-tuning or augmenting training data with local imagery is crucial for accurate inference in new geographies.
  • Challenges in annotating "decay": Certain features like façades and weeds are more difficult to annotate and detect consistently, highlighting the need for advanced techniques like image segmentation for future work.
  • Potential for bias in computer vision: The authors raise an important ethical consideration regarding potential biases in computer vision models, particularly concerning skin color and clothing, and advocate for diverse training data and masking strategies.

Key Results

The study successfully developed an urban quality index that can be applied at the street segment level.

  • Performance across cities: The model achieved a mean Average Precision ([email protected]) of approximately 22% for both San Francisco and Mexico City. However, performance dropped to 3.6% for South Bend, indicating challenges in generalizing to more suburban or rural settings with different urban feature distributions.
  • Feature-specific performance: Detection performance varied across object classes. Well-defined objects like barred windows performed relatively well ([email protected] of 0.478), while features like façades and weeds showed poorer performance.
  • Temporal analysis: The method was applied to track changes over time in three case studies:
  • Tenderloin, San Francisco (2009-2021): Showed a general deterioration in urban quality, with increased detection of tents and graffiti.
  • Mexico City (2016-2017): Visualized changes in urban features and their correlation with urban improvement projects.
  • South Bend, Indiana (2011-2019): Demonstrated the model's ability to capture revitalization efforts, particularly in façade quality, despite lower overall performance.

Limitations and Future Work

  • GSV data limitations: The availability and frequency of GSV imagery can vary significantly by location and time, impacting the completeness and comparability of temporal analyses. The authors suggest caution when interpreting maps based on irregular imagery updates.
  • Model generalizability: While the model shows promise across cities, its performance is significantly lower in less dense, suburban, or rural settings like South Bend. Future work should focus on augmenting training data and adapting methodologies for such contexts.
  • Annotation challenges: The difficulty in annotating certain features (e.g., façades, weeds) suggests that image segmentation might be a more suitable approach for capturing these classes comprehensively.
  • Bias in computer vision: The authors highlight the potential for bias related to demographic features and advocate for further research into ethical considerations and robust annotation strategies.
  • Expanding the definition of urban quality: Future work aims to move beyond decay indicators to include features that enhance positive urban experiences, such as greenery and building aesthetics. The methodology is intended to be adaptable to different urban contexts and evolving criteria for urban quality.

Summary updated

Abstract

The quality of the urban environment is crucial for societal well-being. Yet, measuring and tracking the quality of urban environment, their evolution, and spatial disparities is difficult due to the amount of on-the-ground data needed to capture these patterns. The growing availability of street view images presents new prospects in identifying urban features. However, the reliability and consistency of these methods across different locations and time remains largely unexplored. We aim to develop a comprehensive index of urban quality and change at the street segment level using Google Street View (GSV) imagery. We focus on eight object classes that indicate urban decay or contribute to an unsightly urban space, such as potholes, graffiti, garbage, tents, barred or broken windows, discolored or dilapidated façades, weeds, and utility markings. We train an object detection model on a dataset of images from different cities and assess the performance of these urban indices. We evaluate the effectiveness of this method in various urban contexts over time and discuss its potential for urban planning and public policy. We demonstrate the use of these indices in three applications: the Tenderloin in San Francisco, the Doctores and Historic Center neighborhoods in Mexico City, and South Bend, Indiana.
BibTeX
@inproceedings{vallebueno2023measuring,
  author = {Andrea Vallebueno and Yong Suk Lee},
  title = {{Measuring urban quality and change through the detection of physical attributes of decay}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-44551-3},
}
Figure