Simulating a Hybrid Acquisition System for UAV Platforms

Bashar Alsadik, Fabio Remondino, Francesco NexDrones 2022 · DOI

TL;DR

This paper introduces a simulated hybrid UAV system combining multi-view cameras and a LiDAR scanner to improve 3D mapping efficiency and data quality, especially for complex urban environments. By developing a dedicated flight planning tool and simulating data acquisition, the research demonstrates that this integrated approach can achieve high point densities and adequate facade coverage in a single pass, reducing flight time and costs.

What Problem Does This Paper Address?

Traditional aerial 3D mapping often relies on single-camera systems or separate LiDAR and camera acquisitions. Single-camera systems struggle to capture detailed facade information without complex, multi-pass flight plans, leading to increased flight time, costs, and data processing challenges. While hybrid systems combining cameras and LiDAR exist, they are often expensive or not optimized for comprehensive data capture, particularly for vertical structures. This paper addresses the need for a more efficient, cost-effective, and comprehensive hybrid UAV system capable of capturing rich 3D data, including detailed facade information, in a single flight.

Methods

The core of this research is the simulation of a hybrid UAV acquisition system. The authors designed a system integrating a multi-beam LiDAR (Ouster OS1-32) with two types of commercially available multi-view camera systems: the MAPIR Survey3 and the SenseFly SODA. The camera setup is based on a "Maltese cross" configuration, with four cameras tilted at oblique angles and one nadir camera, aiming to capture facades effectively.

A key contribution is the development of a prototype UAV flight planning tool built in MATLAB. This tool allows users to define mission parameters such as flying height, speed, overlap percentages, and camera tilt angles. It simulates flight paths and estimates output parameters like Ground Sample Distance (GSD), number of flight strips, waypoints, and LiDAR point density. The tool can simulate the integration of different LiDAR sensors (Ouster OS1-32 and Quanergy M8) and camera configurations.

For data processing, simulated multi-view images were rendered in Blender and then processed using Agisoft Metashape. This involved bundle block adjustment for camera pose estimation and Dense Image Matching (DIM) for generating dense point clouds. LiDAR data was simulated along the same flight paths. The final analysis focused on comparing the point densities and accuracies of the individual sensor outputs and the integrated hybrid system.

Datasets

The paper does not use real-world datasets for training or evaluation. Instead, it relies on simulated data. Two simulated urban scenarios were used:

  1. Launceston, Tasmania: A free 3D model of the city was used as the basis for simulating flight paths over an area of 150 x 170 m². Five ground control points (GCPs) and seven checkpoints (CPs) were virtually distributed for accuracy evaluation.
  2. Dortmund, Germany: A simulated area of interest (AOI) of 100 x 200 m² was used, based on data from the ISPRS/EuroSDR benchmark. This scenario employed higher image overlap and slower flight speeds to mimic more realistic 3D modeling missions. Six GCPs and nine CPs were virtually placed for assessment.

Key Insights and Ideas

  • Multi-view cameras are crucial for facade capture: The "Maltese cross" configuration with oblique cameras is essential for capturing detailed information on building facades, which single nadir cameras miss.
  • Integrated LiDAR and photogrammetry offer complementary strengths: LiDAR provides accurate elevation and geometric data, while multi-view photogrammetry adds rich texture and color information, filling gaps and improving overall data completeness.
  • A dedicated flight planning tool is vital: The developed tool simplifies the complex task of planning hybrid UAV missions, ensuring optimal sensor coverage and data quality by considering parameters for both cameras and LiDAR.
  • Low-cost components can yield high-quality results: The research demonstrates that using relatively affordable cameras (MAPIR Survey3, SenseFly SODA) and a capable LiDAR (Ouster OS1-32) can result in dense and accurate 3D models, making advanced mapping more accessible.
  • Optimized flight parameters significantly impact density: Higher overlap and slower flight speeds (as seen in the Dortmund scenario) lead to denser point clouds from both photogrammetry and LiDAR, though this also increases acquisition time.

Key Results

  • High point density: With a flight speed of 5 m/s and 80% image overlap, an average LiDAR point density of up to 1500 pts/m² was achieved with adequate facade coverage in one-pass flight strips.
  • Accuracy:
    • For the SenseFly-based system in Launceston, total RMSE was 4.3 mm for GCPs and 4.6 mm for CPs.
  • For the MAPIR-based system in Launceston, total RMSE was 4.5 mm for GCPs and 3.8 mm for CPs.
  • For the SenseFly-based system in Dortmund, total RMSE was 2.9 mm for GCPs and 4.1 mm for CPs.
  • For the MAPIR-based system in Dortmund, total RMSE was 3.3 mm for GCPs and 5.7 mm for CPs.
  • Facade coverage: The hybrid system demonstrated significant improvement in capturing vertical structures compared to individual DIM or LiDAR point clouds, as visualized in density profiles on building facades.
  • Efficiency: The integrated approach aims to reduce flight planning, acquisition duration, and data handling compared to multi-pass oblique imaging or separate acquisitions.

Limitations and Future Work

  • Simulated data: The study relies entirely on simulated data. While the simulation environment (Blender) is robust, real-world conditions (e.g., atmospheric effects, sensor noise, GPS inaccuracies) are not fully represented.
  • No GNSS/IMU integration: The proposed system does not explicitly include GNSS/IMU sensors for direct georeferencing. While the authors mention it could be combined, the current simulation does not account for their impact.
  • Limited sensor types: Only two camera models and one LiDAR model were investigated. Future work could explore other sensor combinations.
  • Smart data fusion: The paper mentions that the integration of image and range data is currently a "pure merging." Future work could focus on "smart data fusion" techniques to further refine the 3D surveying product.
  • Multispectral capabilities: The authors suggest incorporating multispectral cameras to combine geometric data with spectral information for applications like vegetation mapping or 3D NDVI production.
  • Adaptability to different LiDARs: The system's adaptability to various LiDAR types (e.g., Livox, Quanergy M8, Velodyne Puck) is highlighted as a future direction.

Summary updated

Abstract

Currently, there is a rapid trend in the production of airborne sensors consisting of multi-view cameras or hybrid sensors, i.e., a LiDAR scanner coupled with one or multiple cameras to enrich the data acquisition in terms of colors, texture, completeness of coverage, accuracy, etc. However, the current UAV hybrid systems are mainly equipped with a single camera that will not be sufficient to view the facades of buildings or other complex objects without having double flight paths with a defined oblique angle. This entails extensive flight planning, acquisition duration, extra costs, and data handling. In this paper, a multi-view camera system which is similar to the conventional Maltese cross configurations used in the standard aerial oblique camera systems is simulated. This proposed camera system is integrated with a multi-beam LiDAR to build an efficient UAV hybrid system. To design the low-cost UAV hybrid system, two types of cameras are investigated and proposed, namely the MAPIR Survey and the SenseFly SODA, integrated with a multi-beam digital Ouster OS1-32 LiDAR sensor. Two simulated UAV flight experiments are created with a dedicated methodology and processed with photogrammetric methods. The results show that with a flight speed of 5 m/s and an image overlap of 80/80, an average density of up to 1500 pts/m2 can be achieved with adequate facade coverage in one-pass flight strips.
BibTeX
@inproceedings{alsadik2022simulating,
  author = {Bashar Alsadik and Fabio Remondino and Francesco Nex},
  title = {{Simulating a Hybrid Acquisition System for UAV Platforms}},
  booktitle = {Drones},
  year = {2022},
  doi = {10.3390/drones6110314},
}
Figure