High-quality 3D reconstruction of infrastructure using UAVs is essential for inspection, monitoring, and digital twin applications. Traditional flight planning methods rely on predefined paths and often struggle with complex geometries, leading to incomplete models and inefficiencies. This paper evaluates a state-of-the-art autonomous Next Best View (NBV) model, MACARONS (Mapping And Coverage Anticipation with RGB Online Self-Supervision), which enables online, self-supervised 3D reconstruction of large-scale scenes using only a monocular RGB sensor. The MACARONS NBV model autonomously adjusts UAV trajectories in real time based on predictions of unseen scene structure to improve reconstruction accuracy and surface detail recovery. Simulation results demonstrate that the autonomous NBV strategy significantly enhances both reconstruction quality and operational efficiency, showing a 37.5% efficiency raise and coverage percentages up to 90% compared to traditional planning.
Introduction
Creating high-quality "Digital Twins" of infrastructure, such as bridges or buildings, typically requires a UAV to follow a pre-programmed grid path. However, these static paths are often inefficient and struggle with complex geometries, leaving significant gaps in the final 3D model. This research addresses the challenge of Autonomous View Planning. Instead of following a fixed path, the goal is for the UAV to perceive a scene, predict unseen structures, and move to the "Next Best View" (NBV) to fill in missing data. The study specifically evaluates whether the MACARONS framework can meet the rigorous requirements of professional photogrammetry, where high-quality, overlapping images are necessary for accurate 3D point calculation.
Methodology and Framework
The study centers on the MACARONS framework, which operates through three iterative modules to guide the UAV's path planning:
Depth Module: Predicts depth maps from a sequence of monocular RGB images.
Volume Occupancy Module: Builds a 3D representation of the space to identify which areas are "full" (occupied) and which are "empty."
Surface Coverage Gain Module: The decision-making component that selects the next camera pose by calculating which position will reveal the most new surface area.
Architecture of the MACARONS framework showing the three core modules for depth, volume occupancy, and surface coverage gain.
To test the effectiveness of this autonomous approach, the researchers developed a custom evaluation pipeline. Trajectories generated by the MACARONS AI were imported into Blender to render high-resolution synthetic images using realistic camera intrinsics. These images were then processed in Agisoft Metashape to create dense 3D point clouds, which were compared against ground-truth models of two bridges: a stone arch bridge (Bridge A) and a complex steel truss bridge (Bridge B).
The evaluation workflow integrating MACARONS trajectories with Blender rendering and Metashape reconstruction.
The autonomous MACARONS approach was compared against traditional grid-based flight planning, revealing several key insights:
Efficiency Gains: The autonomous agent significantly reduced the number of images required. For the complex truss bridge, the method used only 392 images compared to 1,106 for the traditional method—a reduction of approximately 65%.
Coverage and Accuracy: While traditional planning achieved near-perfect coverage (97.5%–99%), the autonomous method reached 91.5% for the stone bridge and 88% for the truss bridge. The RMS reprojection error remained comparable between both methods (0.84 to 0.85 pixels), indicating high 3D precision.
The Photogrammetric Gap: A critical finding was that current NBV models focus on seeing a surface once, whereas photogrammetry requires "multi-view" coverage (seeing the same spot from multiple angles). The autonomous method resulted in lower "tie point multiplicity," which can reduce the robustness of the final model.
Conclusion and Future Work
While prediction-based NBV offers a massive boost in operational efficiency (37.5% overall), it currently faces hurdles with self-occlusion in complex lattice structures and a lack of multi-view awareness. Future research should focus on "coverage-aware" planning to ensure every point is captured from multiple distinct angles. The authors suggest that a hybrid approach—where a drone follows a rough manual path but uses NBV to fill in complex details—may be the most effective near-term solution for professional infrastructure inspection.
@article{wang2025autonomous,
author = {Ziwen Wang and Bashar Alsadik and Francesco Nex},
title = {{Autonomous UAV 3D Reconstruction using Prediction-Based Next Best View}},
journal = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
year = {2025},
url = {https://openreview.net/forum?id=USZOd6wkvD},
}