Introduction
3D Gaussian Splatting (3DGS) has emerged as a transformative technique for real-time, high-fidelity scene reconstruction. By representing 3D scenes as a collection of anisotropic Gaussian primitives, it achieves impressive rendering speeds and visual quality. However, a persistent challenge in 3DGS is "depth inconsistency." Because the optimization process typically relies solely on photometric loss—comparing rendered pixels to ground truth images—the model can overfit to specific viewpoints. This results in "floaters," or Gaussian primitives placed at incorrect depths that appear correct from training angles but degrade the underlying 3D structure.
This paper introduces a novel framework designed to enforce geometric consistency by integrating physical camera optics and multi-view geometric supervision. By modeling how real lenses produce defocus blur and aligning monocular depth with multi-view feature matching, the authors significantly improve reconstruction quality, particularly in large-scale urban environments.
The Challenge of Depth Inconsistency
In standard 3DGS, the lack of structural "guardrails" often leads to geometric errors in both far-field (distant backgrounds) and near-field (close-up objects) regions. For instance, in a street-view scene, a distant building might be rendered sharply but mathematically positioned much closer to the camera than it is in reality. These errors are not just theoretical; they manifest as visual artifacts when the camera moves along novel trajectories, breaking the immersion of the reconstructed scene.
Proposed Methodology
The authors propose a dual-branch supervision strategy to anchor Gaussian primitives to their correct spatial locations.
Physical Defocus Modeling
Real-world cameras possess a finite Depth-of-Field (DoF), where objects outside the focus plane appear blurred. The size of this blur, known as the Circle of Confusion (CoC), is directly related to the object's depth relative to the lens.
The framework implements a Differentiable Defocus Convolution. By modeling the CoC mathematically, the system can compare rendered blurred images against ground truth data. This creates a "Depth-of-Field Loss" ($L_{dof}$) that provides a physical gradient signal, pushing Gaussians toward depths that match the observed blur. To enhance realism, the authors utilize adaptive kernels, including polygonal shapes to simulate aperture blades and SmoothStep kernels to maintain sharp edges where appropriate.
Hybrid Depth and Geometric Supervision
While monocular depth estimators like Metric3D provide excellent relative detail, they often struggle with absolute metric scale. To solve this, the authors employ LoFTR (Local Feature TRansformer) to identify corresponding points across multiple views. These matches allow for a global and local scale restoration process, aligning monocular depth maps into a single, consistent metric coordinate system. This aligned depth serves as a strong geometric prior ($L_{depth}$ and $L_{geo}$), preventing the formation of floaters.
Gradient-Aware Density Control
Beyond loss functions, the paper improves the 3DGS optimization loop. Instead of standard pruning, they use gradients from the defocus model to identify "structurally critical" Gaussians. This ensures that complex edges and distant structures are preserved during the densification and pruning phases.
Results and Discussion
The model was rigorously evaluated on several benchmarks, including the Waymo Open Dataset, Mip-NeRF 360, and a custom "LibraryDoF" dataset captured with a Canon R10.
On the Waymo dataset, the method achieved a 35.17 PSNR, representing a 0.8 dB improvement over previous state-of-the-art methods like GaussianPro. Qualitatively, the results show significantly cleaner building facades and more coherent vegetation in distant backgrounds. The "LibraryDoF" experiments further demonstrated the model's ability to reconstruct scenes accurately even when training images contain intentional shallow depth-of-field blur, effectively "deblurring" the scene by understanding its 3D geometry.
Conclusion
By bridging the gap between computer graphics and physical optics, this work provides a robust solution to the depth inconsistency problem in 3D Gaussian Splatting. Treating defocus blur as a "compass" for depth optimization, combined with multi-view geometric constraints, allows for the creation of 3D models that are not only visually stunning but also geometrically accurate. Future work may focus on extending this framework to handle temporal consistency in dynamic scenes and more complex depth discontinuities.
Summary updated