Volumetric neural rendering methods, such as traditional Neural Radiance Fields (NeRF), have transformed 3D scene reconstruction but suffer from a fundamental computational bottleneck. To render a single pixel, these architectures must sample hundreds of coordinate points along a camera ray, querying a neural network for the density and color of each point before integrating them. This iterative ray-marching process imposes a linear computational complexity relative to the number of samples, resulting in high latency and making real-time deployment or scaling to massive environments exceptionally difficult.
To overcome this limitation, a powerful paradigm shift has emerged: bypassing volumetric integration entirely by mapping entire rays directly to their final radiance or geometric properties. By treating the scene as a neural light field rather than a density field, the rendering complexity per ray drops from linear to a constant $O(1)$ time, requiring only a single neural network evaluation per pixel.
The Intuition of Ray Space Mapping
The core insight behind this direction is that the final color of a pixel is ultimately a function of the ray passing through it. If a neural network can learn this direct mapping in ray space, it eliminates the need to inspect every individual point along the ray's path.
Volumetric Rendering (NeRF):
Ray Origin ---> [Sample Point 1] ---> [Sample Point 2] ---> ... ---> [Sample Point N] ---> Integrate ---> Pixel Color
(Neural Query) (Neural Query) (Neural Query)
Single-Evaluation Rendering (NeLF):
Ray Parameters (Origin & Direction) -----------------------------------------------------> Neural Pass ---> Pixel Color
As demonstrated by Vincent Sitzmann et al. in Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering, representing a 3D scene as a function that maps oriented rays directly to radiance values allows for real-time performance. This shift completely avoids computationally expensive ray-marching while still preserving the ability to extract underlying scene geometry via analytical derivatives.
To parameterize these rays efficiently, Celong Liu et al. in NeuLF: Efficient Novel View Synthesis with Neural 4D Light Field utilize a 4D light field representation where each ray is defined by its intersection points with two parallel planes. This coordinate formulation allows a lightweight multilayer perceptron to map the ray coordinates directly to color values in a single pass, accelerating inference up to 1000x compared to volumetric alternatives and unlocking interactive frame rates.
Incorporating Geometry and Spatial Context
While mapping a raw ray to a color is fast, the network must still understand where the ray interacts with the scene's underlying geometry to resolve complex visual details. Researchers have introduced various structural priors to guide the single-pass evaluation.
One approach is to anchor the light field to explicit spatial features. Julian Ost et al. introduce this in Neural Point Light Fields, where a sparse point cloud is used to aggregate scene features. A multi-layer perceptron then predicts the final pixel color directly from the aggregated feature vector and the ray direction in a single pass, scaling view synthesis to massive outdoor environments. Similarly, in Boosting Point Clouds Rendering via Radiance Mapping, the authors simplify volumetric rendering by modeling the view-dependent color exclusively at the first intersection between a camera ray and a surface, replacing complex volumetric integrals with a single neural evaluation per pixel.
Other methods bundle or stream line coordinates to optimize the network's capacity:
- Peng Wang et al. propose ProLiF: Progressively-connected Light Field network for efficient view synthesis, which samples multiple points along a ray but processes their concatenated coordinates simultaneously in a single forward pass, outputting densities and colors for all sample points at once to drastically reduce overhead.
- Arnab Dey et al. demonstrate in Mip-NeRF RGB-D: Depth Assisted Fast Neural Radiance Fields that utilizing external depth information to pinpoint surface locations allows a two-stage coarse-to-fine architecture to be compressed into a single, streamlined network.
- For specialized light transport like caustics, Lars Schjøth et al. show in Photon Differential Splatting for Rendering Caustics that storing camera ray information in an "eye path map" eliminates the need for separate photon map data structures, simplifying the rendering pipeline.
Distillation and Mobile Deployment
Training a direct ray-to-color network from scratch can be challenging because the network must learn complex visibility and view-dependent effects without explicit 3D geometric supervision. To solve this, knowledge distillation is frequently used to transfer the rich representations of a pre-trained volumetric model into a fast neural light field.
In R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis, Huan Wang et al. present a method to distill a pre-trained NeRF into a deep neural light field. By utilizing a single-pass deep residual network to map camera rays directly to pixel colors, they achieve a ~30x rendering speedup while maintaining or improving visual fidelity.
This single-pass efficiency makes the technology highly suitable for resource-constrained environments. Junli Cao et al. adapt this paradigm for smartphones in Real-Time Neural Light Field on Mobile Devices, showing that a single forward network pass per ray can deliver high-quality rendering with significantly reduced latency and storage requirements. To generalize across entirely unseen scenes from only a few input images, Qian Li et al. develop a framework in Learning Generalizable Light Field Networks from Few Images that maps ray coordinates directly to color values in ray space while conditioning the network pass on a 3D feature volume, running roughly 100 times faster than traditional generalizable radiance fields.
Animating and Extending Ray-Space Functions
The utility of single-evaluation ray mapping extends beyond static scenes into dynamic environments and direct geometric regression.
To handle motion, Heng Yu et al. introduce DyLiN: Making Light Field Networks Dynamic. Their method deforms rays into a canonical space and "lifts" them into higher dimensions, mapping entire rays directly to color values in a single forward pass. This maintains real-time performance for dynamic scenes, rendering 25 to 71 times faster than point-sampling dynamic radiance fields. For human expressions, Huan Wang et al. develop LightAvatar: Efficient Head Avatar as Dynamic Neural Light Field, mapping camera rays directly to output colors to achieve a 300-fold increase in frame rates compared to its volumetric teacher model.
Dynamic Ray Evaluation:
Input Ray ---> Deformation & Lifting ---> Canonical Ray Space ---> Single Network Pass ---> Dynamic Color/Distance
Finally, the single-evaluation paradigm can be applied to predict geometric distances instead of radiance:
- Ray-to-Surface Distance: Ajay Kumar et al. introduce RayDF: Neural Ray-surface Distance Fields with Multi-view Consistency, which transitions from coordinate-based sampling to direct ray-to-distance regression. A single query per pixel yields rendering speeds over 1,000 times faster than traditional implicit surface methods.
- Directional Distance Functions: In FIRe: Fast Inverse Rendering using Directional and Signed Distance Functions, Ayush Tewari et al. propose the Directional Distance Function (DDF). By predicting the distance from a ray origin to a surface in a single network evaluation, they bypass the need for iterative sphere tracing, accelerating inverse rendering workflows by over 15 times.
Key Foundational Work
Neural Point Light Fields (2021) — contributed 2 concept elements to this direction.
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering (2021) — contributed 2 concept elements to this direction.
Photon Differential Splatting for Rendering Caustics (2014) — contributed 1 concept elements to this direction.