REAL-TIME DRONE MAPPING BASED ON REFERENCE IMAGES FOR VEHICLE FACILITY MONITORING

H. Kim, Seokil Ham, I. Lee2020 · DOI

TL;DR

This paper introduces a novel real-time drone mapping technique that leverages pre-existing reference images to achieve high accuracy (cm-level) in vehicle facility monitoring, overcoming the traditional trade-off between speed and precision. By using reference aerial triangulation (refAT), the method significantly improves both processing time and accuracy compared to direct georeferencing, making it ideal for rapid and efficient infrastructure assessment.

What Problem Does This Paper Address?

Maintaining critical infrastructure like roads and parking lots is essential for public safety and convenience. However, current monitoring methods often suffer from low efficiency. Traditional methods can be disruptive, requiring facilities to be blocked, or are performed at inconvenient times (e.g., late at night) due to traffic, leading to low-quality data due to poor lighting. Drone-based monitoring offers a promising alternative, but existing approaches face a fundamental challenge: a trade-off between processing speed and accuracy. Direct georeferencing is fast but lacks precision, while traditional post-processing methods are accurate but time-consuming. This paper aims to bridge this gap by enabling real-time, high-accuracy mapping of vehicle facilities.

Methods

The core innovation of this paper is the Reference Aerial Triangulation (refAT) method. This approach builds upon standard aerial triangulation (AT) but incorporates pre-established "reference images." These reference images are meticulously processed beforehand to contain accurate keypoints, descriptors, and exterior/interior orientation parameters (EOPs/IOPs).

Here's how it works:

  1. Reference Image Building: A comprehensive set of reference images for the target area is created and processed to extract reliable keypoints, descriptors, and precise EOPs/IOPs.
  2. Reference Aerial Triangulation (refAT): When a new drone image (acquired image) is captured, the system uses the pre-existing reference images to determine the new image's EOPs. This is achieved by:
  3. FOV Query: Quickly identifying relevant reference images based on the acquired image's field of view.
  4. Image Matching: Matching keypoints between the acquired image and the selected reference images.
  5. Constrained Bundle Adjustment: The crucial step where the AT process uses the known EOPs of the reference images as constraints. This significantly improves the accuracy of the estimated EOPs for the acquired image, without requiring ground control points for every new acquisition.
  6. Orthophoto Generation: Once the accurate EOPs for the acquired image are determined, an individual orthophoto can be generated.

This refAT method contrasts with direct georeferencing (DG), which relies solely on onboard GPS/INS data for immediate positioning, and traditional AT, which typically requires extensive ground control points or a dense network of tie points for post-processing.

Datasets

The study utilized two datasets, acquired in Yangpyeong-gun, Gyeonggi-do, Korea:

  • Dataset A: Consisted of 10 drone images, some captured in winter with snow cover. This dataset was used as the "acquired image" set for testing the real-time processing.
  • Dataset B: Comprised 154 drone images, captured in spring. This dataset was processed using Structure from Motion (SfM) and Multi-View Geometry (MVG) to create the "reference images," establishing a dense point cloud and ground control points (GCPs) with an RMSE of 9 cm.

Eight ground control points were surveyed to perform aerial triangulation and positional accuracy analysis.

Key Insights and Ideas

  • Breaking the Speed-Accuracy Trade-off: The most significant insight is the successful demonstration that real-time drone mapping can achieve high (cm-level) accuracy by intelligently leveraging pre-processed reference data. This challenges the long-held assumption that one must sacrifice speed for precision in drone-based photogrammetry.
  • The Power of "Reference" Data: The concept of pre-computing and storing detailed information (keypoints, descriptors, EOPs) from a well-processed reference dataset is a key enabler. This allows for rapid re-use of established geometric and feature information, drastically reducing computation for new acquisitions.
  • Constrained Adjustment for Efficiency: By using the EOPs of reference images as direct constraints in the bundle adjustment, the refAT method effectively guides the estimation process for new images, leading to faster convergence and higher accuracy without the need for extensive manual intervention or new GCPs for every flight.
  • Improved Orthophoto Quality: Qualitative analysis revealed that refAT-generated orthophotos exhibit fewer "seamlines" and better overall registration compared to those produced by direct georeferencing, making them more interpretable for facility identification.

Key Results

  • Processing Time: The refAT method achieved an average processing time of 8.95 seconds per image for 10 images. This is significantly faster than traditional post-processing but slower than direct georeferencing (average 1.49 seconds per image).
  • Positional Accuracy:
    • refAT achieved an RMSE of 3.4 cm for the overall XY accuracy.
  • Direct Georeferencing (DG) achieved a much lower accuracy of 22.5 m.
  • When comparing individual orthoimages, refAT showed a mean error of approximately 11 cm, with a standard deviation of about 10 cm, indicating good relative accuracy and registration. DG's positional accuracy was significantly worse, with errors around 75 cm and a standard deviation of 4 m.
  • Qualitative Improvement: Orthophotos generated by refAT showed better visual consistency and fewer seamlines, making facility identification easier.

Limitations and Future Work

  • Processing Time vs. DG: While refAT is faster than traditional post-processing, it is still slower than direct georeferencing. The authors acknowledge that further optimization is needed to close this gap.
  • Reference Data Preparation: The initial creation of the reference dataset is a significant undertaking. The paper suggests that more detailed classification and calculation of participating reference images, as well as the integration of Digital Elevation Models (DEMs), could further improve both processing time and accuracy.
  • Generalizability: While tested in a specific rural area, the method's performance across diverse terrains and facility types would be a valuable area for future research.
  • Automation: Further automation in reference image selection and matching could enhance the overall workflow.

Summary updated

Abstract

Facilities such as road, parking lots play an important role in our lives nowadays. Damage to such a vehicle facility can cause human injury, as well as inconvenience and cost. To prevent this, facility monitoring is performed periodically, but the current monitoring method is low efficiency by blocking the facility or performing it late at night. In order to increase the efficiency of monitoring, research using images, especially drone images, was conducted. However, when using a drone image, there is a trade-off relationship between accuracy and processing time. In this study, we propose a real-time drone mapping based on reference images for efficient vehicle facility monitoring. The real-time drone mapping based on the reference image is composed of reference images build, aerial triangulation (AT) based on reference images (refAT), and orthophoto generation. The refAT refers to a method of performing AT by using a reference images as reference data. We compared the processing time and processing accuracy of direct georeferencing and refAT. We built 154 drone reference images in the target area. The refAT showed a processing time of about 8.95 seconds and an accuracy of 3.4 cm, and the direct georeferencing method showed a processing time of about 1.49 seconds and an accuracy of 22.5 m. If the method of this study is used for facility monitoring, it is expected that the efficiency of monitoring will be improved with speed and accuracy.
BibTeX
@inproceedings{kim2020real,
  author = {H. Kim and Seokil Ham and I. Lee},
  title = {{REAL-TIME DRONE MAPPING BASED ON REFERENCE IMAGES FOR VEHICLE FACILITY MONITORING}},
  booktitle = {˜The},
  year = {2020},
  doi = {10.5194/isprs-archives-xliii-b2-2020-43-2020},
}
Figure