Depth-Consistent 3D Gaussian Splatting via Physical Defocus Modeling and Multi-View Geometric Supervision

Yu Deng, Baozhu Zhao, Junyan Su, Xiaohan Zhang, Qi Liu

Introduction

3D Gaussian Splatting (3DGS) has emerged as a transformative technique for real-time, high-fidelity scene reconstruction. By representing 3D scenes as a collection of anisotropic Gaussian primitives, it achieves impressive rendering speeds and visual quality. However, a persistent challenge in 3DGS is "depth inconsistency." Because the optimization process typically relies solely on photometric loss—comparing rendered pixels to ground truth images—the model can overfit to specific viewpoints. This results in "floaters," or Gaussian primitives placed at incorrect depths that appear correct from training angles but degrade the underlying 3D structure.

This paper introduces a novel framework designed to enforce geometric consistency by integrating physical camera optics and multi-view geometric supervision. By modeling how real lenses produce defocus blur and aligning monocular depth with multi-view feature matching, the authors significantly improve reconstruction quality, particularly in large-scale urban environments.

The Challenge of Depth Inconsistency

In standard 3DGS, the lack of structural "guardrails" often leads to geometric errors in both far-field (distant backgrounds) and near-field (close-up objects) regions. For instance, in a street-view scene, a distant building might be rendered sharply but mathematically positioned much closer to the camera than it is in reality. These errors are not just theoretical; they manifest as visual artifacts when the camera moves along novel trajectories, breaking the immersion of the reconstructed scene.

Proposed Methodology

The authors propose a dual-branch supervision strategy to anchor Gaussian primitives to their correct spatial locations.

The proposed framework integrating physical defocus modeling and multi-view geometric supervision.

Physical Defocus Modeling

Real-world cameras possess a finite Depth-of-Field (DoF), where objects outside the focus plane appear blurred. The size of this blur, known as the Circle of Confusion (CoC), is directly related to the object's depth relative to the lens.

Physical modeling of the Circle of Confusion (CoC) based on thin lens optics.

The framework implements a Differentiable Defocus Convolution. By modeling the CoC mathematically, the system can compare rendered blurred images against ground truth data. This creates a "Depth-of-Field Loss" ($L_{dof}$) that provides a physical gradient signal, pushing Gaussians toward depths that match the observed blur. To enhance realism, the authors utilize adaptive kernels, including polygonal shapes to simulate aperture blades and SmoothStep kernels to maintain sharp edges where appropriate.

Hybrid Depth and Geometric Supervision

While monocular depth estimators like Metric3D provide excellent relative detail, they often struggle with absolute metric scale. To solve this, the authors employ LoFTR (Local Feature TRansformer) to identify corresponding points across multiple views. These matches allow for a global and local scale restoration process, aligning monocular depth maps into a single, consistent metric coordinate system. This aligned depth serves as a strong geometric prior ($L_{depth}$ and $L_{geo}$), preventing the formation of floaters.

Gradient-Aware Density Control

Beyond loss functions, the paper improves the 3DGS optimization loop. Instead of standard pruning, they use gradients from the defocus model to identify "structurally critical" Gaussians. This ensures that complex edges and distant structures are preserved during the densification and pruning phases.

Results and Discussion

The model was rigorously evaluated on several benchmarks, including the Waymo Open Dataset, Mip-NeRF 360, and a custom "LibraryDoF" dataset captured with a Canon R10.

On the Waymo dataset, the method achieved a 35.17 PSNR, representing a 0.8 dB improvement over previous state-of-the-art methods like GaussianPro. Qualitatively, the results show significantly cleaner building facades and more coherent vegetation in distant backgrounds. The "LibraryDoF" experiments further demonstrated the model's ability to reconstruct scenes accurately even when training images contain intentional shallow depth-of-field blur, effectively "deblurring" the scene by understanding its 3D geometry.

Conclusion

By bridging the gap between computer graphics and physical optics, this work provides a robust solution to the depth inconsistency problem in 3D Gaussian Splatting. Treating defocus blur as a "compass" for depth optimization, combined with multi-view geometric constraints, allows for the creation of 3D models that are not only visually stunning but also geometrically accurate. Future work may focus on extending this framework to handle temporal consistency in dynamic scenes and more complex depth discontinuities.

Summary updated

BibTeX
@article{deng2026depth,
  author = {Yu Deng and Baozhu Zhao and Junyan Su and Xiaohan Zhang and Qi Liu},
  title = {{Depth-Consistent 3D Gaussian Splatting via Physical Defocus Modeling and Multi-View Geometric Supervision}},
  journal = {Neural Networks},
  year = {2026},
  doi = {10.1016/j.neunet.2025.108320},
}
Figure