N-BVH: Neural ray queries with bounding volume hierarchies

Philippe Weier, Alexander Rath, Élie Michel, Iliyan Georgiev, Philipp Slusallek, Tamy Boubekeur

TL;DR

N-BVH is a hybrid data structure that marries classical ray-tracing acceleration with neural compression. By replacing the deep, memory-heavy leaves of a standard Bounding Volume Hierarchy (BVH) with a lightweight neural model, it achieves up to 42× compression of 3D geometry while allowing path tracers to query intersections, normals, and appearance attributes directly and efficiently.

What Problem Does This Paper Address?

In modern computer graphics, the sheer scale of 3D scenes—comprising millions of polygons and high-resolution textures—often exceeds the available memory (VRAM) on even the most powerful GPUs. While neural representations like NeRFs or Signed Distance Fields (SDFs) offer incredible compression, they are typically designed for point queries (e.g., "What is the density at coordinate X?").

To find a surface intersection with these models, a renderer must "ray march" (step along the ray many times), which is computationally expensive and hard to integrate into standard path-tracing pipelines. The challenge is creating a representation that is both compact and capable of answering ray queries (e.g., "Where does this ray hit, and what does the surface look like there?") in a single pass.

Methods

The authors introduce N-BVH, which consists of two main components:

  1. A Shallow BVH: Instead of a deep tree that goes all the way down to individual triangles, N-BVH uses a "cut" of a standard BVH. The leaf nodes of this shallow tree are "neural nodes" representing clusters of geometry.
  2. Neural Ray Query Pipeline: Each neural node is backed by a multi-resolution hash grid and a tiny Multi-Layer Perceptron (MLP).
  3. Probing: When a ray enters a neural leaf's bounding box, the system samples a few points along the ray's path within that box.
  4. Feature Extraction: It collects features from the hash grid at these points and feeds them into the MLP.
  5. Output: The MLP predicts visibility, the 1D distance to the intersection along the ray, the surface normal, and albedo.

The construction is error-driven: the algorithm starts with a single root node and iteratively splits the nodes with the highest reconstruction error until a target memory budget or node count is reached. This ensures neural capacity is focused on the most complex parts of the scene.

Datasets

The researchers evaluated N-BVH on several high-complexity 3D assets and scenes:

  • Standard Graphics Assets: Chess, Bonzai, Statuette, and City Block.
  • Complex Scans: The Exhibition scene featuring a high-resolution mammoth scan from the Smithsonian Institute.
  • Production-Scale Scenes: The Andalusian Room and assets from the Moana Island Scene (Bay Cedar).
  • Unstructured Geometry: The Rover scene, used to test performance on non-manifold or "messy" geometry.

Key Insights and Ideas

  • 1D is Easier than 3D: A core "aha" moment is that learning the 1D distance along a ray is significantly more accurate and easier for a network than predicting 3D coordinates directly. By constraining the search to the ray-box intersection interval, the mapping becomes much simpler.
  • BVH as a Spatial Filter: The BVH isn't just for speed; it acts as a spatial filter that ensures the neural network only has to learn signals in the immediate vicinity of the geometry, preventing "wasted" capacity on empty space.
  • Seamless Hybridization: Because N-BVH leaves return the same data as a triangle intersection (distance, normal, albedo), a renderer can mix neural assets and classical triangle assets in the same scene without changing the shading logic.
  • Adaptive LoD: The tree-cut optimization naturally provides Levels of Detail (LoD). A coarser cut (fewer nodes) results in a faster but more approximate render, while a deeper cut improves fidelity.

Key Results

  • Massive Compression: The model achieved a 42× compression rate for the geometry it represented. For entire scenes (including non-neural parts), compression was roughly 13×.
  • Memory Efficiency: Complex scenes that originally required gigabytes of storage were compressed into a few dozen megabytes. For example, the Exhibition scene was reduced from 2.69 GB to roughly 180 MB.
  • Rendering Speed: While slower than hardware-accelerated triangle tracing (roughly 2–4× slower due to the software overhead of MLP inference), it remains fast enough for interactive use and is significantly faster than previous neural ray-marching techniques.
  • Visual Fidelity: Using the FLIP error metric, the authors showed that N-BVH provides a faithful approximation of the original geometry, capturing fine details in normals and textures that traditional simplification methods (like edge collapse) often lose at high compression ratios.

Limitations and Future Work

  • Convexity Assumption: The model currently works best when the geometry inside a neural node is relatively "simple" (mostly convex or concave). Very complex, overlapping geometry within a single leaf can lead to artifacts in secondary ray offsets.
  • Static Scenes: Like most neural representations, N-BVH requires a training phase (a few minutes). It does not currently support dynamic, deforming geometry without re-training.
  • Software Bottleneck: The current implementation is fully software-based in CUDA. The authors suggest that future hardware with dedicated "neural cores" or fused ray-tracing/tensor units could drastically close the performance gap with classical ray tracing.
  • Adaptive Sampling: Currently, the system uses a fixed number of sample points per ray. Moving to an adaptive sampling scheme (more points for complex areas) could further optimize the speed-to-quality ratio.

Summary updated

Abstract

Neural representations have shown spectacular ability to compress complex signals in a fraction of the raw data size. In 3D computer graphics, the bulk of a scene’s memory usage is due to polygons and textures, making them ideal candidates for neural compression. Here, the main challenge lies in finding good trade-offs between efficient compression and cheap inference while minimizing training time. In the context of rendering, we adopt a ray-centric approach to this problem and devise <Formula format="inline"><TexMath><?TeX $\mathcal {N}$?></TexMath><AltText>Math 1</AltText><File name="siggraphconferencepapers24-70-inline1" type="svg"/></Formula>-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D. Our compact model is learned from the input geometry and substituted for it whenever a ray intersection is queried by a path-tracing engine. While prior neural compression methods have focused on point queries, ours proposes neural ray queries that integrate seamlessly into standard ray-tracing pipelines. At the core of our method, we employ an adaptive BVH-driven probing scheme to optimize the parameters of a multi-resolution hash grid, focusing its neural capacity on the sparse 3D occupancy swept by the original surfaces. As a result, our <Formula format="inline"><TexMath><?TeX $\mathcal {N}$?></TexMath><AltText>Math 2</AltText><File name="siggraphconferencepapers24-70-inline2" type="svg"/></Formula>-BVH can serve accurate ray queries from a representation that is more than an order of magnitude more compact, providing faithful approximations of visibility, depth, and appearance attributes. The flexibility of our method allows us to combine and overlap neural and non-neural entities within the same 3D scene and extends to appearance level of detail.
BibTeX
@inproceedings{weier2024n,
  author = {Philippe Weier and Alexander Rath and Élie Michel and Iliyan Georgiev and Philipp Slusallek and Tamy Boubekeur},
  title = {{N-BVH: Neural ray queries with bounding volume hierarchies}},
  booktitle = {Siggraph (conference Paper Track)},
  year = {2024},
  doi = {10.1145/3641519.3657464},
}

Loading related content...

Figure