Introduction: The Registration Gap
High-quality 3D reconstruction of large-scale buildings typically requires a continuous sequence of images to establish spatial relationships. In practical scenarios, however, data is often bifurcated into two extremes: high-altitude aerial footage from drones and street-level photography from ground-based cameras.
The significant disparity in perspective and scale between these two viewpoints creates a "registration gap." Standard Structure-from-Motion (SfM) algorithms, such as COLMAP, frequently fail to find matching features across such drastic changes in elevation. Without successful camera registration, these datasets remain unlinked, making a unified 3D reconstruction impossible.
Methodology: Iterative Extrapolation via 3DGS
The DRAGON (DRone And GrOuNd Rendering) framework bridges this gap by leveraging 3D Gaussian Splatting (3DGS). Rather than attempting to match drone and ground images directly, DRAGON employs an iterative extrapolation pipeline to "hallucinate" intermediate views.
The process begins by training a 3DGS model on the drone imagery, which is typically easy to register internally. The model then renders synthetic images at a slightly lower elevation. These "bridge" images are added to the dataset and registered using standard SfM tools. By repeating this process—descending in elevation and registering new synthetic views—the algorithm eventually reaches the ground-level perspective.
To ensure the quality of these extrapolated views, the authors utilize perceptual regularization via DreamSim and OpenCLIP. These deep-learning losses prioritize semantic and structural consistency over pixel-perfect matching, preventing the model from becoming blurry or generating "floaty" artifacts during the extrapolation process.
The Buildings-NVS Dataset
To evaluate the framework, the researchers introduced Buildings-NVS, a semi-synthetic dataset created using Google Earth Studio.
The dataset includes 9 iconic landmarks, such as the Sydney Opera House and the Eiffel Tower. Each scene is structured with five distinct elevation levels (Drone, Mid 1, Mid 2, Mid 3, and Ground), providing a ground-truth sequence to validate the algorithm's ability to bridge the registration gap.
Key Results
DRAGON demonstrates a significant improvement over traditional methods:
- Registration Success: While standard SfM tools often fail to link drone and ground views, DRAGON achieves a 97.47% registration rate with an extremely low positional error of 0.01m.
- Reconstruction Quality: The framework produces sharper and more complete 3D models compared to baselines trained on single-elevation data.
- Ablation Insights: The inclusion of perceptual regularization was found to be critical in maintaining structural integrity and removing "sky-like" artifacts in extrapolated regions.
Conclusion and Future Directions
DRAGON successfully demonstrates that Novel View Synthesis (NVS) can serve as a powerful preprocessing tool for traditional computer vision tasks. By generating synthetic "bridge" data, it enables the registration of previously incompatible datasets. Future work will focus on adapting the framework to real-world noise—such as moving pedestrians and changing weather—and automating the selection of intermediate elevation paths.
Summary updated