DDNeRF: Depth Distribution Neural Radiance Fields

David Dadon, Ohad Fried, Yacov Hel-Or

Abstract

Neural Radiance Fields (NeRF) have transformed 3D scene representation but suffer from high computational costs due to inefficient sampling. Standard hierarchical sampling relies on a coarse model that treats density as a piecewise-constant function, limiting depth resolution. We present DDNeRF (Depth Distribution Neural Radiance Fields), which models density along a ray as a mixture of truncated Gaussians. By predicting internal distribution parameters, DDNeRF achieves superior image quality and geometric accuracy with up to 3x fewer samples or iterations compared to standard methods.

Introduction

Neural Radiance Fields (NeRF) require querying a neural network at numerous points along a ray to render a single pixel. To optimize this, most implementations use a two-stage hierarchical sampling strategy: a "coarse" model identifies regions of interest, and a "fine" model concentrates samples there. However, the coarse model typically assumes density is constant within each interval (piecewise-constant), which restricts the fine model's resolution to $1/n^2$ of the scene depth. This often results in wasted samples in empty space or missed fine details unless a very high sample count is used.

Methods

DDNeRF addresses these limitations by refining how the coarse model describes volume density.

Internal Distribution Modeling

Instead of predicting a single density value per interval, the DDNeRF coarse model predicts the parameters of a Gaussian distribution—mean ($\mu$) and standard deviation ($\sigma$)—representing the conditional density within that interval.

Comparison of hierarchical sampling between standard NeRF (piecewise-constant) and DDNeRF (Gaussian mixture).

As shown in the comparison above, while standard NeRF uses a coarse model PDF that is piecewise-constant, DDNeRF produces a continuous mixture of Gaussians that more accurately captures points of interest.

Architecture and Mixture of Truncated Gaussians

To ensure density remains within its assigned segment, the model employs truncated Gaussians. The total density along the ray is then represented as a continuous mixture of these distributions. This allows the model to pinpoint surface locations regardless of the initial interval size, effectively breaking the resolution limits of standard NeRF.

The DDNeRF architecture, illustrating the flow from coarse network distribution prediction to fine network sampling.

The architecture incorporates a Distribution Estimation (DE) Loss. This KL-divergence-based loss allows the fine model to supervise the coarse model. As the fine model gains a more accurate understanding of the geometry, it teaches the coarse model to predict better Gaussian parameters, creating a beneficial feedback loop.

Smoothing Factor

To prevent premature convergence on incorrect depths, a smoothing factor ($u$) artificially widens the Gaussians during early training. This factor is gradually reduced, sharpening the focus as the model becomes more confident.

Experimental Results

DDNeRF was evaluated across several datasets, including LLFF, Synthetic 360°, and a complex new "Motorcycle" scene.

  • Efficiency and Quality: DDNeRF consistently outperformed Mip-NeRF across various sampling budgets. For instance, in the "Motorcycle" scene, a DDNeRF model with 32 samples achieved a higher PSNR (20.84) than a Mip-NeRF model using 96 samples (20.80).
  • Training Speed: By requiring fewer samples and iterations to reach target quality, DDNeRF can achieve baseline-equivalent results in approximately one-third of the training time (e.g., 5.5 hours vs. 11.8 hours).
  • Geometric Accuracy: Qualitative analysis shows that DDNeRF produces cleaner 3D point clouds and meshes, significantly reducing "floaters" and noise in complex geometric areas like tires or tool carts.

Conclusion

DDNeRF significantly boosts the efficiency of neural radiance fields by replacing flat density assumptions with more expressive Gaussian distributions. While each iteration carries a slight overhead (~18% slower), the overall reduction in required iterations and samples leads to substantial net gains in training speed and reconstruction quality. Future work will explore integrating these sampling improvements with ultra-fast encoding methods like Instant NGP.

Summary updated

Abstract

The field of implicit neural representation has made significant progress. Models such as neural radiance fields (NeRF) [12], which uses relatively small neural networks, can represent high-quality scenes and achieve state-of-the-art results for novel view synthesis. Training these types of networks, however, is still computationally expensive and the model struggles with real life 360° scenes. In this work, we propose the depth distribution neural radiance field (DDNeRF), a new method that significantly increases sampling efficiency along rays during training, while achieving superior results for a given sampling budget. DDNeRF achieves this performance by learning a more accurate representation of the density distribution along rays. More specifically, the proposed framework trains a coarse model to predict the internal distribution of the transparency of an input volume along each ray. This estimated distribution then guides the sampling procedure of the fine model. Our method allows using fewer samples during training while achieving better output quality with the same computational resources.
BibTeX
@inproceedings{dadon2023ddnerf,
  author = {David Dadon and Ohad Fried and Yacov Hel-Or},
  title = {{DDNeRF: Depth Distribution Neural Radiance Fields}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
  doi = {10.1109/wacv56688.2023.00082},
}
Figure