Single Evaluation Neural Light Fields

20 concepts14 papers

Volumetric neural rendering methods, such as traditional Neural Radiance Fields (NeRF), have transformed 3D scene reconstruction but suffer from a fundamental computational bottleneck. To render a single pixel, these architectures must sample hundreds of coordinate points along a camera ray, querying a neural network for the density and color of each point before integrating them. This iterative ray-marching process imposes a linear computational complexity relative to the number of samples, resulting in high latency and making real-time deployment or scaling to massive environments exceptionally difficult.

To overcome this limitation, a powerful paradigm shift has emerged: bypassing volumetric integration entirely by mapping entire rays directly to their final radiance or geometric properties. By treating the scene as a neural light field rather than a density field, the rendering complexity per ray drops from linear to a constant $O(1)$ time, requiring only a single neural network evaluation per pixel.

The Intuition of Ray Space Mapping

The core insight behind this direction is that the final color of a pixel is ultimately a function of the ray passing through it. If a neural network can learn this direct mapping in ray space, it eliminates the need to inspect every individual point along the ray's path.

Volumetric Rendering (NeRF):
Ray Origin ---> [Sample Point 1] ---> [Sample Point 2] ---> ... ---> [Sample Point N] ---> Integrate ---> Pixel Color
                (Neural Query)        (Neural Query)                 (Neural Query)

Single-Evaluation Rendering (NeLF):
Ray Parameters (Origin & Direction) -----------------------------------------------------> Neural Pass ---> Pixel Color

As demonstrated by Vincent Sitzmann et al. in Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering, representing a 3D scene as a function that maps oriented rays directly to radiance values allows for real-time performance. This shift completely avoids computationally expensive ray-marching while still preserving the ability to extract underlying scene geometry via analytical derivatives.

To parameterize these rays efficiently, Celong Liu et al. in NeuLF: Efficient Novel View Synthesis with Neural 4D Light Field utilize a 4D light field representation where each ray is defined by its intersection points with two parallel planes. This coordinate formulation allows a lightweight multilayer perceptron to map the ray coordinates directly to color values in a single pass, accelerating inference up to 1000x compared to volumetric alternatives and unlocking interactive frame rates.

Incorporating Geometry and Spatial Context

While mapping a raw ray to a color is fast, the network must still understand where the ray interacts with the scene's underlying geometry to resolve complex visual details. Researchers have introduced various structural priors to guide the single-pass evaluation.

One approach is to anchor the light field to explicit spatial features. Julian Ost et al. introduce this in Neural Point Light Fields, where a sparse point cloud is used to aggregate scene features. A multi-layer perceptron then predicts the final pixel color directly from the aggregated feature vector and the ray direction in a single pass, scaling view synthesis to massive outdoor environments. Similarly, in Boosting Point Clouds Rendering via Radiance Mapping, the authors simplify volumetric rendering by modeling the view-dependent color exclusively at the first intersection between a camera ray and a surface, replacing complex volumetric integrals with a single neural evaluation per pixel.

Other methods bundle or stream line coordinates to optimize the network's capacity:

Distillation and Mobile Deployment

Training a direct ray-to-color network from scratch can be challenging because the network must learn complex visibility and view-dependent effects without explicit 3D geometric supervision. To solve this, knowledge distillation is frequently used to transfer the rich representations of a pre-trained volumetric model into a fast neural light field.

In R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis, Huan Wang et al. present a method to distill a pre-trained NeRF into a deep neural light field. By utilizing a single-pass deep residual network to map camera rays directly to pixel colors, they achieve a ~30x rendering speedup while maintaining or improving visual fidelity.

This single-pass efficiency makes the technology highly suitable for resource-constrained environments. Junli Cao et al. adapt this paradigm for smartphones in Real-Time Neural Light Field on Mobile Devices, showing that a single forward network pass per ray can deliver high-quality rendering with significantly reduced latency and storage requirements. To generalize across entirely unseen scenes from only a few input images, Qian Li et al. develop a framework in Learning Generalizable Light Field Networks from Few Images that maps ray coordinates directly to color values in ray space while conditioning the network pass on a 3D feature volume, running roughly 100 times faster than traditional generalizable radiance fields.

Animating and Extending Ray-Space Functions

The utility of single-evaluation ray mapping extends beyond static scenes into dynamic environments and direct geometric regression.

To handle motion, Heng Yu et al. introduce DyLiN: Making Light Field Networks Dynamic. Their method deforms rays into a canonical space and "lifts" them into higher dimensions, mapping entire rays directly to color values in a single forward pass. This maintains real-time performance for dynamic scenes, rendering 25 to 71 times faster than point-sampling dynamic radiance fields. For human expressions, Huan Wang et al. develop LightAvatar: Efficient Head Avatar as Dynamic Neural Light Field, mapping camera rays directly to output colors to achieve a 300-fold increase in frame rates compared to its volumetric teacher model.

Dynamic Ray Evaluation:
Input Ray ---> Deformation & Lifting ---> Canonical Ray Space ---> Single Network Pass ---> Dynamic Color/Distance

Finally, the single-evaluation paradigm can be applied to predict geometric distances instead of radiance:

Key Foundational Work

Neural Point Light Fields (2021) — contributed 2 concept elements to this direction.
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering (2021) — contributed 2 concept elements to this direction.
Photon Differential Splatting for Rendering Caustics (2014) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_direct_ray_to_color_mapping_bbf417d2,
  title = {direct ray-to-color mapping},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/direct-ray-to-color-mapping/rdr_bbf417d293e94b83a80ccd5b00d71255},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_direct_ray_to_color_mapping_bbf417d2_v1,
  title = {direct ray-to-color mapping},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/direct-ray-to-color-mapping/rdr_bbf417d293e94b83a80ccd5b00d71255/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}

See something wrong? Report a mistake