Abstract: In our previous research, we examined whether minimally trained crowd workers could find, categorize, and assess sidewalk accessibility problems using Google Street View (GSV) images. This poster paper presents a first step towards combining automated methods (e.g., machine vision-based curb ramp detectors) in concert with human computation to improve the overall scalability of our approach.
Introduction
Navigating urban environments is a significant challenge for individuals with mobility impairments. High-quality, up-to-date data on sidewalk accessibility—such as the presence and quality of curb ramps—is notoriously difficult to collect via traditional manual audits, which are slow and expensive. While previous research demonstrated that remote crowd workers could identify these features using Google Street View (GSV), the process remains labor-intensive. This paper explores a hybrid "human-in-the-loop" approach to address this scalability gap by using computer vision to perform the initial detection and humans to verify the results.
Methodology
The researchers developed a multi-stage pipeline to automate the detection process:
- Data Acquisition & Masking: The system scrapes GSV panoramas and 3D point cloud data. To minimize false positives (e.g., detecting ramps on rooftops), 3D data and OpenStreetMap metadata are used to create image masks that isolate only sidewalk and intersection areas.
- Automated Detection: A linear Support Vector Machine (SVM) is trained using Histograms of Oriented Gradients (HOG). A multi-scale sliding window approach is then used to find potential curb ramps.
- Crowdsourced Verification: The top 10 detections per intersection, ranked by classifier confidence, are sent to Amazon Mechanical Turk. Workers answer a simple binary question: "Is this a curb ramp?" To ensure accuracy, each label is verified by three different workers to reach a majority-vote consensus.
Evaluation and Results
The study focused on two areas in Washington DC: Mt. Pleasant (a semi-urban mixed area) and the vicinity of the White House (a dense urban area). The researchers manually labeled 437 curb ramps across 118 intersections to create a ground truth dataset.
The impact of human verification was dramatic. The researchers tested two definitions of a "correct" label based on overlap with ground truth:
- 10% Overlap: The automated detector's initial precision of 22.7% jumped to 90.4% after human verification.
- 50% Overlap: Under this stricter criterion, precision rose from 9.2% to 43.9%.
While precision saw a massive boost, recall (the ability to find all existing ramps) remained low. The classifier often missed ramps with low visual contrast or those viewed from difficult angles. However, the verification process itself proved highly efficient: 26 workers completed 1,770 verifications with a median time of just 27.3 seconds per set of 10 labels.
Discussion and Future Work
A core insight of this work is that verifying a detection is significantly faster and cheaper than finding and labeling one from scratch. By shifting the human's role from "searcher" to "editor," the system becomes much more scalable.
Future work aims to address the current limitations in recall by utilizing multiple camera viewpoints and integrating top-down satellite imagery. Additionally, the researchers plan to implement active learning, using the human-verified data to iteratively re-train and improve the SVM detector, creating a virtuous cycle of increasing machine intelligence.
Summary updated