Autonomous UAV 3D Reconstruction using Prediction-Based Next Best View

Ziwen Wang, Bashar Alsadik, Francesco NexISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2025 · OpenReview

Abstract

High-quality 3D reconstruction of infrastructure using UAVs is essential for inspection, monitoring, and digital twin applications. Traditional flight planning methods rely on predefined paths and often struggle with complex geometries, leading to incomplete models and inefficiencies. This paper evaluates a state-of-the-art autonomous Next Best View (NBV) model, MACARONS (Mapping And Coverage Anticipation with RGB Online Self-Supervision), which enables online, self-supervised 3D reconstruction of large-scale scenes using only a monocular RGB sensor. The MACARONS NBV model autonomously adjusts UAV trajectories in real time based on predictions of unseen scene structure to improve reconstruction accuracy and surface detail recovery. Simulation results demonstrate that the autonomous NBV strategy significantly enhances both reconstruction quality and operational efficiency, showing a 37.5% efficiency raise and coverage percentages up to 90% compared to traditional planning.

Introduction

Creating high-quality "Digital Twins" of infrastructure, such as bridges or buildings, typically requires a UAV to follow a pre-programmed grid path. However, these static paths are often inefficient and struggle with complex geometries, leaving significant gaps in the final 3D model. This research addresses the challenge of Autonomous View Planning. Instead of following a fixed path, the goal is for the UAV to perceive a scene, predict unseen structures, and move to the "Next Best View" (NBV) to fill in missing data. The study specifically evaluates whether the MACARONS framework can meet the rigorous requirements of professional photogrammetry, where high-quality, overlapping images are necessary for accurate 3D point calculation.

Methodology and Framework

The study centers on the MACARONS framework, which operates through three iterative modules to guide the UAV's path planning:

  1. Depth Module: Predicts depth maps from a sequence of monocular RGB images.
  2. Volume Occupancy Module: Builds a 3D representation of the space to identify which areas are "full" (occupied) and which are "empty."
  3. Surface Coverage Gain Module: The decision-making component that selects the next camera pose by calculating which position will reveal the most new surface area.
Architecture of the MACARONS framework showing the three core modules for depth, volume occupancy, and surface coverage gain.

Evaluation Pipeline

To test the effectiveness of this autonomous approach, the researchers developed a custom evaluation pipeline. Trajectories generated by the MACARONS AI were imported into Blender to render high-resolution synthetic images using realistic camera intrinsics. These images were then processed in Agisoft Metashape to create dense 3D point clouds, which were compared against ground-truth models of two bridges: a stone arch bridge (Bridge A) and a complex steel truss bridge (Bridge B).

The evaluation workflow integrating MACARONS trajectories with Blender rendering and Metashape reconstruction.

Results and Discussion

The autonomous MACARONS approach was compared against traditional grid-based flight planning, revealing several key insights:

  • Efficiency Gains: The autonomous agent significantly reduced the number of images required. For the complex truss bridge, the method used only 392 images compared to 1,106 for the traditional method—a reduction of approximately 65%.
  • Coverage and Accuracy: While traditional planning achieved near-perfect coverage (97.5%–99%), the autonomous method reached 91.5% for the stone bridge and 88% for the truss bridge. The RMS reprojection error remained comparable between both methods (0.84 to 0.85 pixels), indicating high 3D precision.
  • The Photogrammetric Gap: A critical finding was that current NBV models focus on seeing a surface once, whereas photogrammetry requires "multi-view" coverage (seeing the same spot from multiple angles). The autonomous method resulted in lower "tie point multiplicity," which can reduce the robustness of the final model.

Conclusion and Future Work

While prediction-based NBV offers a massive boost in operational efficiency (37.5% overall), it currently faces hurdles with self-occlusion in complex lattice structures and a lack of multi-view awareness. Future research should focus on "coverage-aware" planning to ensure every point is captured from multiple distinct angles. The authors suggest that a hybrid approach—where a drone follows a rough manual path but uses NBV to fill in complex details—may be the most effective near-term solution for professional infrastructure inspection.

Summary updated

Abstract

Articles | Volume X-2/W2-2025 ArticleMetricsRelated articles Articles | Volume X-2/W2-2025 https://doi.org/10.5194/isprs-annals-X-2-W2-2025-207-2025 © Author(s) 2025. This work is distributed under the Creative Commons Attribution 4.0 License. https://doi.org/10.5194/isprs-annals-X-2-W2-2025-207-2025 © Author(s) 2025. This work is distributed under the Creative Commons Attribution 4.0 License. Articles | Volume X-2/W2-2025 ArticleMetricsRelated articles 29 Oct 2025 | 29 Oct 2025 Autonomous UAV 3D Reconstruction using Prediction-Based Next Best View Ziwen Wang, Bashar Alsadik, and Francesco Nex Ziwen Wang × Department of Earth Observation Science, Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Hallenweg 6, 7522 NH Enschede, the Netherlands Bashar Alsadik https://orcid.org/0000-0002-9425-9325 × Department of Earth Observation Science, Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Hallenweg 6, 7522 NH Enschede, the Netherlands Francesco Nex https://orcid.org/0000-0002-5712-6902 × Department of Earth Observation Science, Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Hallenweg 6, 7522 NH Enschede, the Netherlands Keywords: NBV, Autonomous, MACARONS, 3D Reconstruction, Self-supervision, Blender Abstract. High-quality 3D reconstruction of infrastructure using UAVs is essential for inspection, monitoring, and digital twin applications. Traditional flight planning methods rely on predefined paths and often struggle with complex geometries, leading to incomplete models and inefficiencies. This paper evaluates a state-of-the-art autonomous Next Best View (NBV) of MACARONS model (Mapping And Coverage Anticipation with RGB Online Self-Supervision), which enables online, self-supervised 3D reconstruction of large-scale scenes using only a monocular RGB sensor. The MACARONS NBV model autonomously adjusts UAV trajectories in real time based on predictions of unseen scene structure to improve reconstruction accuracy and surface detail recovery. Despite its advantages, a key limitation is its lack of consideration for camera coverage percentage from a photogrammetric perspective, which makes it challenging to consistently obtain an informative point cloud. The simulation results demonstrate that the autonomous NBV strategy significantly enhances both reconstruction quality and operational efficiency. To evaluate its effectiveness, we applied the MACARONS NBV model to two open-access 3D bridge models. The generated camera trajectories were imported into Blender, where we rendered high-resolution images using realistic camera intrinsics to overcome the limitations of the low-resolution depth predictions. From these images, we reconstructed point clouds and compared them to those produced by a traditional flight planning approach, as well as to the ground truth models. The comparison highlights the added value of autonomous view planning for accurate and efficient UAV-based 3D reconstruction. The two experiments showed a high coverage percentage of 88 % compared to the ground truth and 90% compared to traditional flight planning based on a 37.5% efficiency raise. This work highlights the potential and current limitations of prediction-based NBV in UAV photogrammetry and motivates further research into integrating coverage-aware planning. Download & links Article (PDF, 1419 KB) Download & links Article (1419 KB) Metadata XML BibTeX EndNote Share document.addEventListener("DOMContentLoaded", function () { const mobileShareElement = document.querySelector(".mobile-native-share"); if (navigator.share) { // Native sharing is available if (mobileShareElement) { mobileShareElement.style.display = "block"; } } else { // Native sharing is NOT available if (mobileShareElement) { mobileShareElement.style.display = "none"; } } }); How to cite. Wang, Z., Alsadik, B., and Nex, F.: Autonomous UAV 3D Reconstruction using Prediction-Based Next Best View, ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., X-2/W2-2025, 207–214, https://doi.org/10.5194/isprs-annals-X-2-W2-2025-207-2025, 2025.
BibTeX
@article{wang2025autonomous,
  author = {Ziwen Wang and Bashar Alsadik and Francesco Nex},
  title = {{Autonomous UAV 3D Reconstruction using Prediction-Based Next Best View}},
  journal = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
  year = {2025},
  url = {https://openreview.net/forum?id=USZOd6wkvD},
}
Figure