Abstract
Neural Radiance Fields (NeRF) have revolutionized novel view synthesis but remain computationally expensive due to inefficient hierarchical sampling strategies. Standard NeRF implementations rely on a coarse model that produces a piecewise-constant probability density function (PDF), requiring the fine model to sample uniformly across broad intervals. We introduce DDNeRF (Depth Distribution Neural Radiance Fields), which significantly enhances efficiency by enabling the coarse model to predict a continuous Gaussian distribution of density within each sampling interval. By employing truncated Gaussian mixtures and a novel Distribution Estimation loss, DDNeRF concentrates samples on critical scene structures. Our evaluation demonstrates that DDNeRF achieves superior image quality and depth estimation with a fraction of the sampling budget required by state-of-the-art baselines.
Introduction
The primary bottleneck in standard NeRF architectures is the hierarchical sampling process. To render a ray, a "coarse" network first identifies regions of high density, followed by a "fine" network that performs the final rendering. However, the coarse model's output is typically a piecewise-constant PDF—essentially a series of blocks. This "limited depth resolution" means that for $n$ samples, the resolution cannot exceed $1/n^2$ of the scene depth. Consequently, if an object occupies only a small portion of a sampling block, the fine model must still sample the entire block uniformly, leading to wasted computation or missed fine details.
Methods
DDNeRF extends the MipNeRF framework by introducing internal distribution prediction. Instead of a single opacity value per interval, the coarse network is modified to predict the parameters of a Gaussian distribution—mean ($\mu$) and standard deviation ($\sigma$)—representing the density distribution within that specific interval.
Truncated Gaussian Mixtures
To maintain the independence of interval calculations required for computational efficiency, DDNeRF employs Truncated Gaussians. The final PDF used for sampling is constructed as a mixture of these truncated distributions, weighted by the total opacity of each interval. This ensures that density predictions do not bleed across interval boundaries while providing a much more informative guide for the fine sampling stage.
Distribution Estimation (DE) Loss
A key innovation is the Distribution Estimation (DE) loss. This loss function utilizes Kullback-Leibler (KL) divergence to align the coarse model’s predicted Gaussian parameters with the actual density distribution subsequently discovered by the fine model. This feedback loop allows the fine model to effectively "teach" the coarse model how to better estimate depth distributions. Additionally, an uncertainty smoothing factor is applied during early training to prevent premature convergence on incorrect depths, gradually sharpening the focus as the model gains confidence.
Results and Discussion
DDNeRF was evaluated across several challenging domains, including LLFF, Synthetic 360°, and large-scale environments from Tanks and Temples. The model consistently outperformed MipNeRF across PSNR, SSIM, and LPIPS metrics, particularly in low-sample regimes.
- Sampling Efficiency: On the "LEGO" scene, DDNeRF with only 32 samples achieved a PSNR of 31.53, surpassing MipNeRF even when the latter used 96 samples on similar scenes.
- Visual Fidelity: Qualitative results show that DDNeRF captures fine details—such as motorcycle spokes and thin tool cart wheels—that are often blurred or lost by standard models at low budgets.
- Geometric Accuracy: Notably, DDNeRF's coarse model produces depth maps that are qualitatively and quantitatively closer to the fine model's output than standard NeRF, indicating a more physically grounded internal representation early in the training process.
Conclusion
DDNeRF demonstrates that making the coarse model "smarter" rather than just denser leads to significant gains in rendering efficiency and quality. While the model shows some limitations in handling the far backgrounds of vast, unbounded scenes, it sets a new standard for efficient hierarchical sampling in bounded environments. Future work will explore hybridizing DDNeRF’s foreground distribution estimation with background-warping techniques to handle infinite horizons.
Summary updated