N-BVH: Neural ray queries with bounding volume hierarchies

Philippe Weier, Alexander Rath, Élie Michel, Iliyan Georgiev, Philipp Slusallek, Tamy Boubekeur

Abstract

N-BVH is a hybrid data structure that integrates classical ray-tracing acceleration with neural compression. By replacing the memory-intensive leaf nodes of a standard Bounding Volume Hierarchy (BVH) with lightweight neural models, N-BVH achieves up to 42× compression of 3D geometry. This architecture allows path tracers to query intersections, normals, and appearance attributes directly and efficiently, bridging the gap between compact neural representations and high-performance rendering pipelines.

Introduction

In modern computer graphics, the scale and complexity of 3D scenes—often comprising millions of polygons—frequently exceed the available video memory (VRAM) on consumer and professional GPUs. While neural representations like Neural Radiance Fields (NeRFs) offer impressive compression, they are typically designed for point queries, requiring computationally expensive ray marching to find surface intersections. N-BVH addresses this challenge by creating a representation that is both compact and capable of answering direct ray queries (e.g., "Where does this ray hit?") in a single pass.

Methodology

N-BVH consists of two primary components: a shallow BVH and a neural ray query pipeline. Instead of a deep tree reaching individual triangles, N-BVH uses a "cut" of a standard BVH where the leaf nodes are "neural nodes" representing clusters of geometry.

Neural ray query pipeline illustrating ray sampling within cut-node bounding boxes.

Each neural node is backed by a multi-resolution hash grid and a compact Multi-Layer Perceptron (MLP). When a ray enters a neural leaf's bounding box, the system samples points along the ray's path within that box. Features are extracted from the hash grid and fed into the MLP to predict visibility, 1D distance to intersection, surface normals, and albedo. A key insight of this work is that learning 1D distance along a ray is significantly more accurate and easier for a network than predicting 3D coordinates directly.

The construction is error-driven: the algorithm iteratively splits nodes with the highest reconstruction error until a target memory budget is reached, ensuring neural capacity is focused on the most complex parts of the scene.

Datasets and Evaluation

The researchers evaluated N-BVH across a variety of high-complexity 3D assets:

  • Standard Graphics Assets: Chess, Bonzai, Statuette, and City Block.
  • Complex Scans: The Exhibition scene, featuring a high-resolution mammoth scan.
  • Production-Scale Scenes: The Andalusian Room and assets from the Moana Island Scene.
  • Unstructured Geometry: The Rover scene, used to test performance on non-manifold or "messy" geometry.

Key Results

N-BVH achieved a 42× compression rate for the geometry it represented, with overall scene compression (including non-neural components) reaching approximately 13×. For example, the Exhibition scene was reduced from 2.69 GB to roughly 180 MB.

High-fidelity rendering of a complex interior scene achieved using N-BVH.

While software-based MLP inference is currently 2–4× slower than hardware-accelerated triangle tracing, it remains fast enough for interactive use and is significantly faster than previous neural ray-marching techniques. Visual fidelity, measured by the FLIP error metric, shows that N-BVH provides a faithful approximation of the original geometry, capturing fine details in normals and textures that traditional simplification methods often lose.

Discussion and Future Work

Current limitations include a convexity assumption for geometry within neural nodes and the requirement for a training phase, which currently limits the approach to static scenes. Future work may explore hardware acceleration via dedicated neural cores or fused ray-tracing units to close the performance gap with classical methods. Additionally, adaptive sampling schemes could further optimize the speed-to-quality ratio by allocating more sample points to complex regions.

Summary updated

BibTeX
@inproceedings{weier2024n,
  author = {Philippe Weier and Alexander Rath and Élie Michel and Iliyan Georgiev and Philipp Slusallek and Tamy Boubekeur},
  title = {{N-BVH: Neural ray queries with bounding volume hierarchies}},
  booktitle = {Siggraph (conference Paper Track)},
  year = {2024},
  doi = {10.1145/3641519.3657464},
}
Figure