Neural 3D Object Reconstruction with Small-Scale Unmanned Aerial Vehicles

Álmos Veres-Vitályos, Genís Castillo Gómez-Raya, Filip Lemic, Daniel Johannes Bugelnig, Bernhard Rinner, Sergi Abadal, Xavier Costa-Pérez

Abstract

Small Unmanned Aerial Vehicles (UAVs) exhibit immense potential for navigating indoor and hard-to-reach areas, yet their significant constraints in payload and autonomy have largely prevented their use for complex tasks like high-quality 3-Dimensional (3D) reconstruction. To overcome this challenge, we introduce a novel system architecture that enables fully autonomous, high-fidelity 3D scanning of static objects using UAVs weighing under 100 grams. Our core innovation lies in a dual-reconstruction pipeline that creates a real-time feedback loop between data capture and flight control. A near-real-time (near-RT) process uses Structure from Motion (SfM) to generate an instantaneous pointcloud of the object. The system analyzes the model quality on the fly and dynamically adapts the UAV's trajectory to intelligently capture new images of poorly covered areas. This ensures comprehensive data acquisition. For the final, detailed output, a non-real-time (non-RT) pipeline employs a Neural Radiance Fields (NeRF)-based Neural 3D Reconstruction (N3DR) approach, fusing SfM-derived camera poses with precise Ultra Wide-Band (UWB) location data to achieve superior accuracy. We implemented and validated this architecture using Crazyflie 2.1 UAVs. Our experiments, conducted in both single- and multi-UAV configurations, conclusively show that dynamic trajectory adaptation consistently improves reconstruction quality over static flight paths.

Introduction

Small Unmanned Aerial Vehicles (UAVs) are the "insects" of the robotics world—agile, able to squeeze into tight spaces, and relatively inexpensive. However, their tiny size comes with a massive trade-off: they lack the payload capacity for heavy LiDAR sensors and the onboard compute for complex autonomy. Traditionally, high-quality 3D reconstruction has been the domain of much larger, heavier platforms. This research introduces a system that allows sub-100-gram drones to autonomously perform high-fidelity 3D scans by combining real-time feedback with cutting-edge neural rendering.

Problem Statement: Miniaturization vs. Autonomy

3D reconstruction usually requires expensive, heavy LiDAR sensors or a human pilot carefully flying a camera-equipped drone to ensure every angle is captured. Small UAVs are perfect for indoor or hard-to-reach areas, but they are severely limited by battery life and processing power. If a tiny drone follows a pre-programmed "static" path, it might miss crucial angles, resulting in models with holes or blurry patches. Conversely, processing high-fidelity models onboard is currently impossible. This work addresses the gap between miniaturization and autonomy, enabling drones to "know" if they have captured enough data and adjust their flight path accordingly.

Figure 1: Comparison of UAV-based 3D reconstruction works by object size and UAV weight, highlighting the niche addressed by this research.

Proposed Methodology

The core innovation is a dual-reconstruction pipeline that splits the workload between the drone and a base station:

  1. Near-Real-Time (near-RT) Pipeline: As the drone flies, it streams low-resolution images to a base station. The station uses Structure from Motion (SfM) to build a rough, "instantaneous" point cloud. The system calculates a "coverage score" for different slices of the object; if a slice is poorly covered, the system dynamically updates the drone's trajectory to capture more data.
  2. Non-Real-Time (non-RT) Pipeline: Post-flight, all captured images are processed using Nerfacto (a Neural Radiance Fields or NeRF variant) to create a dense, volumetric 3D representation.
  3. Localization Fusion: To ensure accurate camera poses, the system fuses Ultra Wide-Band (UWB) indoor positioning data with visual poses derived from SfM.
Figure 2: Overview of the dual-reconstruction system architecture, illustrating the feedback loop between the near-RT SfM pipeline and the UAV flight control.

The hardware setup utilizes the Crazyflie 2.1 equipped with an AI-Deck, providing a low-power camera and a Wi-Fi module to stream data back to a PC.

Experimental Evaluation

The system was evaluated using two custom 3D-printed reference objects: a small block featuring high-contrast text ("i2cat" and "UPC" logos) and a 90cm tall anatomical model of the human gastrointestinal system printed in transparent polymer. These objects were tested using both Black-and-White (BW) and RGB camera modules in single- and multi-UAV configurations.

Key Results and Discussion

  • Dynamic vs. Static Trajectories: Dynamic trajectory adaptation improved the Peak Signal-to-Noise Ratio (PSNR) of near-RT models by approximately 1 dB compared to static paths, proving that intelligent flight leads to superior data quality.
  • High Fidelity Reconstruction: The final NeRF-based models achieved high accuracy, with Structural Similarity Index (SSIM) scores often exceeding 0.96 for the small object and 0.98 for the large anatomical model.
  • Latency and Efficiency: The near-RT pipeline could merge new images into the rough model in 23–34 seconds, enabling meaningful real-time updates to the flight plan.
  • Multi-UAV Synergy: Using multiple drones improved reconstruction stability by providing a higher density of viewpoints, aiding NeRF convergence.

Conclusion

This work demonstrates a scalable and autonomous solution that unlocks the potential of miniaturized UAVs for fine-grained 3D reconstruction in constrained environments. While current limitations include a dependency on offboard processing and UWB sensitivity to interference, the transition from passive cameras to intelligent sensors marks a significant advancement. Future work will explore Gaussian Splatting (GS) for faster rendering and integrated onboard obstacle avoidance for navigation in unknown, cluttered spaces.

Summary updated

Abstract

Small Unmanned Aerial Vehicles (UAVs) exhibit immense potential for navigating indoor and hard-to-reach areas, yet their significant constraints in payload and autonomy have largely prevented their use for complex tasks like high-quality 3-Dimensional (3D) reconstruction. To overcome this challenge, we introduce a novel system architecture that enables fully autonomous, high-fidelity 3D scanning of static objects using UAVs weighing under 100 grams. Our core innovation lies in a dual-reconstruction pipeline that creates a real-time feedback loop between data capture and flight control. A near-real-time (near-RT) process uses Structure from Motion (SfM) to generate an instantaneous pointcloud of the object. The system analyzes the model quality on the fly and dynamically adapts the UAV's trajectory to intelligently capture new images of poorly covered areas. This ensures comprehensive data acquisition. For the final, detailed output, a non-real-time (non-RT) pipeline employs a Neural Radiance Fields (NeRF)-based Neural 3D Reconstruction (N3DR) approach, fusing SfM-derived camera poses with precise Ultra Wide-Band (UWB) location data to achieve superior accuracy. We implemented and validated this architecture using Crazyflie 2.1 UAVs. Our experiments, conducted in both single- and multi-UAV configurations, conclusively show that dynamic trajectory adaptation consistently improves reconstruction quality over static flight paths. This work demonstrates a scalable and autonomous solution that unlocks the potential of miniaturized UAVs for fine-grained 3D reconstruction in constrained environments, a capability previously limited to much larger platforms.
BibTeX
@misc{veresvitalyos2025neural,
  author = {Álmos Veres-Vitályos and Genís Castillo Gómez-Raya and Filip Lemic and Daniel Johannes Bugelnig and Bernhard Rinner and Sergi Abadal and Xavier Costa-Pérez},
  title = {{Neural 3D Object Reconstruction with Small-Scale Unmanned Aerial Vehicles}},
  year = {2025},
  eprint = {2509.12458},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.12458},
}
Figure