Abstract
Standard Neural Radiance Fields (NeRFs) typically represent 3D scenes using isotropic point features, assuming that volume density and latent features are independent of the viewing direction. However, this assumption fails to account for the directional nature of sampling intervals along camera rays, leading to "shape-radiance ambiguity," blurring, and aliasing. We propose a "plug-and-play" anisotropic neural representation that models density and features as view-dependent functions using Spherical Harmonics (SH). By factorizing representations into view-independent and view-dependent components and applying a novel anisotropy regularization loss, our method significantly improves rendering quality across various state-of-the-art NeRF architectures.
Introduction: The Problem of Directional Ambiguity
Standard NeRF models assign a single volume density ($\sigma$) and latent feature ($e$) to each point in space. These values are isotropic, meaning they remain constant regardless of the observer's perspective. In reality, a "point" sample in volume rendering represents a small interval along a ray. In complex scenes—such as those with fine textures, semi-transparent surfaces, or intricate geometry—the effective density of this interval can vary significantly depending on the viewing angle.
Forcing an isotropic representation on these directional intervals creates a fundamental ambiguity: the model cannot distinguish whether a visual effect is due to geometry (density) or appearance (color). This often results in "fuzzy" surfaces, blurry textures, and "floaters" in the reconstructed scene.
Proposed Method: Anisotropic Neural Representation
To address these limitations, we introduce an anisotropic representation where geometry itself is view-dependent.
SH-Guided Features
Instead of predicting a single scalar for density, our primary MLP predicts coefficients ($k$ and $W$) for a set of Spherical Harmonics (SH). These basis functions allow us to represent signals on a sphere efficiently. For a given ray with direction $d$, the model queries these functions to obtain a view-dependent density $\sigma(d, k)$ and latent feature $e(d, W)$. This allows the representation to capture the "internal structure" of a sampling interval.
Anisotropy Regularization
Allowing geometry to be view-dependent introduces a risk of overfitting, where the model might create "fake" geometry to match training images from specific angles. To ensure geometric consistency, we introduce a point-wise regularization loss, $\mathcal{L}_{aniso}$. This loss penalizes the energy of higher-order SH components, encouraging the model to maintain an isotropic representation unless the training data provides strong evidence for anisotropy.
Architecture Agnostic Integration
Our module is designed to be "plug-and-play." We demonstrate its versatility by integrating it into four prominent NeRF frameworks: Nerfacc, K-Planes, Tri-Mip, and Zip-NeRF.
Experimental Evaluation
We evaluated our method on two benchmark datasets:
- Blender Dataset: 8 synthetic scenes used for high-detail reconstruction testing.
- Mip-360 Dataset: 7 real-world unbounded scenes with complex backgrounds and intricate objects.
Results
The addition of the anisotropic module consistently improved performance across all baselines. On the Blender dataset, integrating the module into Nerfacc increased the PSNR from 33.06 to 34.08. On the real-world Mip-360 dataset, Zip-NeRF saw an improvement from 28.67 to 29.14 PSNR.
Qualitatively, the method excels at capturing fine details, such as the sharp edges of foliage, realistic specular reflections on metallic surfaces, and clear textures on distant objects. The regularization effectively reduces "shape-radiance ambiguity," leading to cleaner depth maps and fewer geometric artifacts.
Conclusion and Future Work
By moving beyond the isotropic assumption, our anisotropic neural representation provides a more accurate model for volume rendering in complex scenes. While the use of Spherical Harmonics introduces a minor computational overhead, the gains in rendering quality and geometric fidelity are substantial. Future work will explore alternative basis functions and optimization techniques for real-time applications on mobile hardware.
Summary updated