Abstract
The ability to retrieve and analyze recent images of critical infrastructure assets is beneficial for regular monitoring, post-disaster assessment, or preparing for a service call. Given a high-quality image of an asset, several recently developed deep learning models can automatically assess the state of the infrastructure. However, obtaining such an image automatically remains an open question. Enterprise imaging initiatives, such as Google Street View, permit the viewing of road-adjacent images, given geographic coordinates. The spatial resolution of such systems is excellent, although the temporal resolution varies from months to years. We have recently forecast the emergence of on-demand imaging using instrumented vehicles that would permit more recent or frequent imaging of locations of interest. However, the challenge remains to retrieve a high-quality image of an asset of interest, free from obstructions and imaging artifacts. We here propose a pipeline to retrieve recent images of an asset given an imaging source, GPS coordinates, and an asset class. Object detection is used to automatically identify the asset of interest and to detect obstructions or imaging artifacts. If necessary, additional images are requested for surrounding locations to provide multiple views of the asset of interest culminating in an image free from artifacts. The pipeline is demonstrated using two critical infrastructure asset classes (utility poles and street lights) and two image sources (Streetview and a repository of dashcam video). Robust performance is observed, resulting in correct asset identification and imaging in 76.5% of cases (up from 54.5%), while requiring an average of 1.47 images per asset to achieve a high-quality image free from obstructions and artifacts. The proposed pipeline will be of interest to disaster response teams, utilities, and other critical infrastructure asset managers.
Introduction: The "Garbage In, Garbage Out" Challenge
Automated infrastructure monitoring—using AI to spot storm damage, vegetation encroachment, or equipment wear—is a major goal for utilities and disaster response teams. However, computer vision models require high-quality, unobstructed images to work reliably. While massive repositories like Google Street View (GSV) or municipal dashcam feeds exist, they are "non-specific" and frequently contain imaging artifacts that degrade model performance.
Common artifacts include:
- Stitching errors: Objects appearing doubled or disconnected due to panorama processing.
- Environmental noise: Sun glare or motion blur.
- Framing issues: The asset being partially cropped by the edge of the photo.
Currently, obtaining a "clean" shot often requires manual searching or dispatching a crew, which negates the efficiency of automated monitoring. This research proposes an automated pipeline to "search for a good photo" by iteratively adjusting the viewpoint based on real-time artifact detection.
Proposed Methodology
The researchers developed a 6-step algorithmic pipeline that treats image acquisition as an iterative process. The logic flow is designed to detect both the target asset and any occluding artifacts, moving the virtual camera if the initial view is compromised.
- Query: Given a GPS coordinate and asset class (e.g., "utility pole"), the pipeline requests an initial image from a source (GSV API or a dashcam repository).
- Segmentation: The system runs an instance segmentation model to identify the Asset of Interest and any Occluding Classes (blur, glare, doubling artifacts, or image margins).
- Overlap Check: It calculates if the asset's mask overlaps with any artifact masks.
- Selection: If multiple assets are present, it selects the most prominent one (the largest mask).
- Decision & Reposition: If the image is "dirty" (has artifacts), the pipeline calculates a new viewpoint. For GSV, it moves the virtual camera orthogonal to the original heading; for dashcams, it moves backward in time through the video frames.
- Iteration: This repeats until a clean image is found or a movement limit is reached.
The researchers evaluated several AI architectures for this pipeline, including fine-tuned YOLOv8 models (ranging from nano to extra-large) and GroundedSAM (a zero-shot combination of GroundingDINO and the Segment Anything Model).
Experimental Setup and Datasets
The pipeline was tested against two primary datasets:
- Surrey GSV Dataset: 3,456 images derived from 1,000 randomly chosen utility poles in Surrey, BC, manually annotated for assets and artifacts.
- Ottawa Dashcam Dataset: 4,675 frames extracted from dashcam footage taken in Ottawa, ON, covering various weather conditions.
- Infrastructure Metadata: Open data from the cities of Surrey and Ottawa provided ground-truth GPS locations for the assets.
Key Results and Insights
The study found that artifacts are a significant bottleneck; in a baseline "nearest image" request, nearly 35% of Surrey utility pole images contained significant artifacts. The proposed pipeline successfully mitigated these issues:
- Improved Success Rate: The pipeline increased the retrieval of "good" images from 54.5% to 76.5% for Surrey utility poles.
- High Efficiency: On average, the system required only 1.47 image requests per asset to find a high-quality shot, minimizing API costs and processing time.
- Model Selection: While the largest YOLOv8 models had the highest raw accuracy, the medium model (YOLOv8m) offered the best balance for the pipeline, achieving high F1 scores for artifact detection without excessive computational overhead.
- Zero-Shot Performance: GroundedSAM performed well without specific training but had a higher false-positive rate, occasionally leading to unnecessary "moves."
Conclusion and Future Work
The proposed pipeline demonstrates a robust method for automating the retrieval of high-quality infrastructure images. By treating image acquisition as an active, iterative process rather than a static query, the system significantly improves the reliability of downstream AI monitoring tasks. Future work will focus on handling transient physical obstructions, such as parked vehicles or heavy snow, and improving performance in dense urban environments where asset clustering can lead to identification errors.
The implementation is available on GitHub.
Summary updated