SIM: A mapping framework for built environment auditing based on street view imagery

Huan Ning, Zhenlong Li, Manzhu Yu, Wenpeng YinarXiv 2025 · DOI · arXiv

Introduction

Built environment auditing—the systematic documentation of urban features such as sidewalks, traffic signs, and trees—is a cornerstone of urban planning and public health research. Traditionally, this process relied on labor-intensive field surveys. While Street View Imagery (SVI) has enabled remote auditing, a significant technical gap remains: most AI tools can detect objects in 2D photos but struggle to determine their precise 3D location or physical dimensions without expensive camera metadata or LiDAR.

The SIM (Street Image Mapping) framework addresses this by providing an open-source toolkit that transforms standard SVI into a high-precision tool for urban auditing. By combining geometric principles with deep learning, it extracts real-world measurements from 2D images, effectively bridging the gap between simple object detection and professional-grade geospatial mapping.

The SIM Framework

The SIM framework is structured as a modular system consisting of a foundational algorithm library and three specialized mapping pipelines designed for distinct auditing tasks.

The SIM framework architecture, comprising a foundational algorithm library and three specialized mapping pipelines.

Specialized Mapping Pipelines

  1. Width Measurement (Ground Objects): This pipeline converts SVI into Bird’s Eye View (BEV) land cover maps. It employs a "scanline" approach and run-length encoding to measure the width of linear features like roads or sidewalks. A critical innovation is the attribute-based filtering system, which automatically identifies and discards measurements corrupted by parked cars or shadows.
The width measurement pipeline workflow, illustrating the transition from raw imagery to encoded geospatial attributes.
  1. 3D Localization (Known Dimensions): Utilizing tacheometric surveying techniques, this pipeline locates objects in 3D space using a single image. By leveraging "prior knowledge"—such as the standard height of a stop sign (0.75m)—the algorithm calculates distance and GPS coordinates based on pixel height and the camera’s Field of View (FoV).
  2. Diameter Measurement (Vertical Objects): This pipeline uses stereo-vision principles to identify the same object (e.g., a tree trunk) in overlapping images from adjacent panoramas. It integrates Grounded-SAM for precise segmentation and DINO v2 for monocular depth estimation to assist in triangulation.

Evaluation and Results

The researchers validated the SIM framework across three distinct case studies:

  • Road Width (Washington, D.C.): Using 157,000 panoramas, the framework achieved a mean error of 0.85 meters (approximately 10% relative error) on roads averaging 9.76m wide.
  • Stop Sign Localization (Maryland): Tested against a dataset of 17,049 signs, the horizontal Root-Mean-Square Error (RMSE) was 2.54 meters, with 65.9% of signs located within 2 meters of their true position.
  • Tree Diameter (Columbia, SC): Validated against 83 manually measured trees, the pipeline showed an average relative error of only 7.5% (roughly a 4cm margin), performing competitively with manual field measurements.

Discussion and Future Work

The SIM framework demonstrates that professional-grade results do not strictly require expensive LiDAR hardware; geometric logic combined with standardized object knowledge can yield practical accuracy. However, challenges remain, including stitching artifacts in panoramas and occlusions from parked cars or dense foliage.

Future development will focus on integrating "open-vocabulary" models. This will allow researchers to map arbitrary objects—such as fire hydrants or trash cans—using simple text prompts, further lowering the barrier to comprehensive urban auditing.

Summary updated

BibTeX
@misc{ning2025sim,
  author = {Huan Ning and Zhenlong Li and Manzhu Yu and Wenpeng Yin},
  title = {{SIM: A mapping framework for built environment auditing based on street view imagery}},
  year = {2025},
  eprint = {2505.24076},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.24076},
}
Figure