TL;DR
N-BVH is a hybrid data structure that marries classical ray-tracing acceleration with neural compression. By replacing the deep, memory-heavy leaves of a standard Bounding Volume Hierarchy (BVH) with a lightweight neural model, it achieves up to 42× compression of 3D geometry while allowing path tracers to query intersections, normals, and appearance attributes directly and efficiently.
What Problem Does This Paper Address?
In modern computer graphics, the sheer scale of 3D scenes—comprising millions of polygons and high-resolution textures—often exceeds the available memory (VRAM) on even the most powerful GPUs. While neural representations like NeRFs or Signed Distance Fields (SDFs) offer incredible compression, they are typically designed for point queries (e.g., "What is the density at coordinate X?").
To find a surface intersection with these models, a renderer must "ray march" (step along the ray many times), which is computationally expensive and hard to integrate into standard path-tracing pipelines. The challenge is creating a representation that is both compact and capable of answering ray queries (e.g., "Where does this ray hit, and what does the surface look like there?") in a single pass.
Methods
The authors introduce N-BVH, which consists of two main components:
- A Shallow BVH: Instead of a deep tree that goes all the way down to individual triangles, N-BVH uses a "cut" of a standard BVH. The leaf nodes of this shallow tree are "neural nodes" representing clusters of geometry.
- Neural Ray Query Pipeline: Each neural node is backed by a multi-resolution hash grid and a tiny Multi-Layer Perceptron (MLP).
- Probing: When a ray enters a neural leaf's bounding box, the system samples a few points along the ray's path within that box.
- Feature Extraction: It collects features from the hash grid at these points and feeds them into the MLP.
- Output: The MLP predicts visibility, the 1D distance to the intersection along the ray, the surface normal, and albedo.
The construction is error-driven: the algorithm starts with a single root node and iteratively splits the nodes with the highest reconstruction error until a target memory budget or node count is reached. This ensures neural capacity is focused on the most complex parts of the scene.
Datasets
The researchers evaluated N-BVH on several high-complexity 3D assets and scenes:
- Standard Graphics Assets: Chess, Bonzai, Statuette, and City Block.
- Complex Scans: The Exhibition scene featuring a high-resolution mammoth scan from the Smithsonian Institute.
- Production-Scale Scenes: The Andalusian Room and assets from the Moana Island Scene (Bay Cedar).
- Unstructured Geometry: The Rover scene, used to test performance on non-manifold or "messy" geometry.
Key Insights and Ideas
- 1D is Easier than 3D: A core "aha" moment is that learning the 1D distance along a ray is significantly more accurate and easier for a network than predicting 3D coordinates directly. By constraining the search to the ray-box intersection interval, the mapping becomes much simpler.
- BVH as a Spatial Filter: The BVH isn't just for speed; it acts as a spatial filter that ensures the neural network only has to learn signals in the immediate vicinity of the geometry, preventing "wasted" capacity on empty space.
- Seamless Hybridization: Because N-BVH leaves return the same data as a triangle intersection (distance, normal, albedo), a renderer can mix neural assets and classical triangle assets in the same scene without changing the shading logic.
- Adaptive LoD: The tree-cut optimization naturally provides Levels of Detail (LoD). A coarser cut (fewer nodes) results in a faster but more approximate render, while a deeper cut improves fidelity.
Key Results
- Massive Compression: The model achieved a 42× compression rate for the geometry it represented. For entire scenes (including non-neural parts), compression was roughly 13×.
- Memory Efficiency: Complex scenes that originally required gigabytes of storage were compressed into a few dozen megabytes. For example, the Exhibition scene was reduced from 2.69 GB to roughly 180 MB.
- Rendering Speed: While slower than hardware-accelerated triangle tracing (roughly 2–4× slower due to the software overhead of MLP inference), it remains fast enough for interactive use and is significantly faster than previous neural ray-marching techniques.
- Visual Fidelity: Using the FLIP error metric, the authors showed that N-BVH provides a faithful approximation of the original geometry, capturing fine details in normals and textures that traditional simplification methods (like edge collapse) often lose at high compression ratios.
Limitations and Future Work
- Convexity Assumption: The model currently works best when the geometry inside a neural node is relatively "simple" (mostly convex or concave). Very complex, overlapping geometry within a single leaf can lead to artifacts in secondary ray offsets.
- Static Scenes: Like most neural representations, N-BVH requires a training phase (a few minutes). It does not currently support dynamic, deforming geometry without re-training.
- Software Bottleneck: The current implementation is fully software-based in CUDA. The authors suggest that future hardware with dedicated "neural cores" or fused ray-tracing/tensor units could drastically close the performance gap with classical ray tracing.
- Adaptive Sampling: Currently, the system uses a fixed number of sample points per ray. Moving to an adaptive sampling scheme (more points for complex areas) could further optimize the speed-to-quality ratio.
Summary updated