Abstract
Horizon-GS is a novel 3D Gaussian Splatting (3D-GS) framework designed to bridge the gap between aerial and street-level imagery for large-scale urban reconstruction. By employing a coarse-to-fine training strategy and a balanced camera distribution, it achieves high-fidelity rendering and seamless transitions across unprecedented scale changes, outperforming existing state-of-the-art methods in both visual quality and geometric accuracy.
1. Introduction
Reconstructing large-scale urban environments typically presents a trade-off: capturing global structures and rooftops via aerial views versus capturing fine-grained textures like storefronts and sidewalks via street-level views. Existing neural rendering methods, such as NeRF or standard 3D-GS, struggle to integrate these disparate perspectives into a single, cohesive model.
The primary challenges identified are gradient conflicts during densification—where street views might prune Gaussians essential for aerial views—and data imbalance, where the higher density of street-level data leads to overfitting and the loss of global context.
2. Methodology
Horizon-GS introduces a unified 3D representation through several key innovations designed to handle the "tug-of-war" between different viewpoints.
2.1 Coarse-to-Fine Training
To resolve geometric conflicts, the training is bifurcated. Stage 1 utilizes aerial images to establish a stable "skeleton" of the scene's coarse geometry. Stage 2 freezes this core structure and incorporates street-level images to populate high-frequency details. This ensures that global geometry is established before local noise from ground-level occlusions can degrade the structure.
2.2 Balanced Optimization and LOD
A balanced camera distribution sampling strategy ensures aerial perspectives maintain influence during optimization, preventing street-view bias. Furthermore, a multi-resolution Level of Detail (LOD) system, built on an octree structure, activates different Gaussian layers based on the viewpoint. This enables sharp close-up details and efficient distant rendering, achieving real-time speeds of approximately 51 FPS even on massive environments.
3. Experimental Results
The framework was evaluated on the new Horizon-GS dataset, comprising synthetic and real-world scenes (Road and Park), as well as existing benchmarks like MatrixCity and UC-GS.
Horizon-GS consistently outperforms baselines such as Scaffold-GS and Hierarchical-3DGS. In synthetic tests, it achieved a PSNR of 30.59 for aerial views and 23.80 for street views, a significant improvement over standard 3D-GS (25.44 and 21.81, respectively). Qualitatively, the model produces cleaner reconstructed meshes and handles drastic view changes—such as diving from a bird's-eye view to the sidewalk—without the popping or blurring artifacts common in single-domain models.
4. Conclusion
Horizon-GS marks a significant advancement in large-scale urban reconstruction by successfully unifying aerial and ground-level perspectives. While the system remains dependent on accurate initial Structure-from-Motion (SfM) calibration and primarily targets static scenes, its hierarchical LOD and coarse-to-fine training provide a robust foundation for future developments in dynamic urban modeling and foundation-model integration.