TL;DR
Sidewalk Stewards is a visual analytics system that combines automated computer vision with human expertise to repair flawed sidewalk network maps. By using a human-in-the-loop approach, the system allows users to efficiently identify and fix topological errors—like gaps caused by tree cover—while simultaneously retraining a local model to improve future extractions with as few as 20 manual corrections.
What Problem Does This Paper Address?
While we have highly accurate digital maps for cars, pedestrian infrastructure remains a "data desert." Automated pipelines like Tile2Net have recently made it possible to extract sidewalk networks from high-resolution aerial imagery at city scales. However, these models hit a "hard ceiling" due to environmental occlusions: tree canopies, building shadows, and ambiguous pavement textures lead to disconnected segments and "dangling" edges.
For urban planners and routing engines (especially those serving people with disabilities), a sidewalk network with a 5-meter gap is as useless as no network at all. Currently, fixing these errors requires tedious manual labor in standard Geographic Information Systems (GIS), which lack the specialized tools to prioritize the most impactful errors or provide real-time feedback on network quality.
Methods
The authors propose a multi-scale visual analytics framework organized into a Macro-Meso-Micro pipeline:
- Ingest & Detect: The system ingests aerial imagery and initial network graphs. It uses a Segment Anything Model (SAM) to delineate surface boundaries and calculates an "uncertainty heatmap" to flag where the automated model likely failed.
- Inspect: Users navigate through coordinated views. A Parallel Coordinates Plot (PCP) allows analysts to filter tiles by specific features (e.g., "high uncertainty" + "high tree cover"). A tile gallery then ranks these candidates for inspection.
- Repair: This is the core innovation. The system provides automated repair suggestions (connecting nearby dangling endpoints) and direct editing tools.
- Human-in-the-Loop Retraining: As the user confirms or rejects fixes, a U-Net (ResNet34 encoder) repair model is retrained in the background. This model learns the specific morphological "quirks" of the local city, allowing it to predict and fix similar errors across the rest of the dataset automatically.
To ground these edits, the system embeds Google Street View directly into the interface, allowing users to verify if a sidewalk exists under a tree canopy without leaving the tool.
Datasets
The researchers validated the system using two primary urban contexts:
- Boston, USA: A dataset of 1,000 tiles was used to develop a typology of common errors (e.g., "Meandering sidewalks," "Vegetated surfaces"). A separate set of 1,151 tiles served as the held-out validation set.
- Recife, Brazil: Used as a case study to test the system’s generalizability to the Global South, where pavement materials and urban density differ significantly from the North American training data.
Key Insights and Ideas
- The Power of Few-Shot Correction: One of the most striking findings is that a very small number of user corrections (just 10 to 20) can significantly boost the performance of the segmentation model for a specific city. This suggests that "local" fine-tuning is more effective than "global" general models for urban infrastructure.
- Error Typology as Scaffolding: By identifying six specific families of errors (like "interrupted sidewalks" at driveways), the authors provide a mental framework that helps analysts work faster. Experts noted that having these "starting points" reduced the psychological burden of auditing thousands of miles of data.
- Context over Geometry: Traditional GIS focuses on geometric precision, but Sidewalk Stewards prioritizes topological continuity. The system understands that for a pedestrian, the connection between two segments is often more important than the exact pixel-perfect placement of the line.
Key Results
- Efficiency Gains: In a comparative user study against QGIS (the industry standard), participants using Sidewalk Stewards found more errors in less time. The average "fixing time" dropped from 283 seconds to 241 seconds per task.
- Higher Completion: Users achieved an 86% completion rate with the new system, compared to 68.4% in the conventional workflow.
- Quantitative Improvement: The repair model trained on 20 corrections improved the revised Recall (rRec) on "very hard" tiles from a dismal 0.05 to 0.51. It also significantly improved routing metrics like Reachability and Average Path Length Similarity (APLS).
Limitations and Future Work
The authors acknowledge that the system still relies heavily on the availability of high-resolution aerial imagery, which may not be available for all municipalities. Furthermore, while the system is excellent at fixing segmentation, it can sometimes increase the count of "dangling endpoints" if the automated post-processing isn't perfectly aligned with the new segments.
Future work includes integrating sunlight access models (to help planners prioritize shaded routes) and exploring the use of LiDAR data to supplement aerial imagery in areas with extreme tree cover.
Check it out: The tool is available at urbantk.org/sidewalk.
Summary updated