Real-Time Volumetric Light Transport Inference without a Path

Ludwic Leonard, Ruediger WestermannarXiv 2026 · arXiv

TL;DR

HaloRT is a novel neural rendering approach that achieves real-time volumetric light transport without performing any path tracing at inference time. By training an attention-augmented U-Net to infer full global illumination from inexpensive primary-ray features, the authors achieve noise-free, high-fidelity cloud rendering in approximately 10ms—orders of magnitude faster than traditional path-traced references.

What Problem Does This Paper Address?

Rendering participating media like clouds, smoke, or fog is notoriously difficult. Unlike surface rendering, where light paths terminate quickly, light in volumes scatters at every point along a ray, turning each pixel into a massive recursive integration problem.

Current state-of-the-art methods rely on Monte Carlo (MC) denoising. These pipelines take a few "noisy" path-traced samples and use neural networks to clean them up. However, even generating those few initial samples is computationally expensive in dense volumes. Furthermore, the resulting images often suffer from temporal instability (flickering) or blurring. The authors ask a radical question: Do we even need path-traced samples at all?

Methods

The core idea of HaloRT is to treat rendering as a deterministic decoding problem. Instead of sampling random paths, the system extracts a compact set of "near-sufficient" features along primary rays and uses a neural network to "infer" the final radiance.

1. Feature Extraction

The system computes several low-cost features for every pixel:

  • Optical Depth: The total density encountered by the ray.
  • Interaction Depths: Multi-scale estimates of where the first scattering events are likely to occur.
  • Transmittance to Light: How much light reaches the first interaction point from the sun/directional source.
  • Environment Illumination: A low-sample estimate of ambient light from the sky.
  • Local Medium Properties: Parameters like albedo (reflectivity) and anisotropy (scattering direction).

2. The HaloRT Architecture

The features are fed into a specialized U-Net called HaloRT.

  • Local Spatial Self-Attention (Halo Attention): Unlike standard convolutions, this allows the network to relate features across different image regions (crucial for multiple scattering) without the quadratic cost of full global attention.
  • Linearity Exploitation: The model uses separate decoders for directional (sun) and environmental (sky) light. Because light transport is linear, these components can be inferred independently and summed, allowing the model to handle arbitrary light intensities and colors without retraining.
The HaloRT pipeline: Features are extracted along primary rays and processed by specialized neural models to produce the final scattered light components.

Datasets

The authors generated a massive synthetic dataset to train the model:

  • Volumes: 500 unique cloud formations from the Cloudy Project dataset.
  • Lighting: 1,000 HDR environment maps synthesized using Stable Diffusion to provide diverse natural and stylized lighting.
  • Scale: A total of 22,000 instances were rendered with 4,096 samples per pixel (spp) to serve as ground-truth references.
  • Evaluation: A held-out set of 15 clouds never seen during training was used for testing.

Key Insights and Ideas

  • Feature Sufficiency: The most significant finding is that primary-ray features are "near-sufficient" for clouds. You don't need to simulate the entire path to predict the final look; the local geometry and visibility to light sources provide enough context for a trained network to fill in the complex multiple-scattering effects.
  • Deterministic Stability: Because the input features are mostly deterministic (or very low-variance), the output is incredibly stable. Traditional denoisers struggle with "boiling" noise; HaloRT produces rock-solid images across frames.
  • Constant-Time Rendering: In traditional path tracing, cost increases with volume density. In HaloRT, the inference cost is tied to image resolution, making performance predictable regardless of how thick the clouds are or how close the camera gets.

Key Results

  • Speed: HaloRT runs in ~10.66 ms at 512x512 resolution on an NVIDIA RTX 5090. This is more than twice as fast as feature-enhanced denoisers (which take ~24ms) and thousands of times faster than path-traced references (which take ~50s).
  • Quality: While traditional denoisers might achieve higher PSNR (a pixel-to-pixel accuracy metric), HaloRT consistently scores better on LPIPS, a metric that better reflects human perceptual quality. It preserves fine details and "rim lighting" effects that denoisers often blur away.
  • Generalization: The model successfully renders clouds and lighting environments it has never seen before, proving it has learned the underlying physics of light transport rather than just memorizing images.
Figure 1: For clouds and environments it has never seen during training, our attention-based neural model reconstructs full volumetric global illumination from inexpensive primary-ray features and environment samples in real-time on an NVIDIA GeForce RTX 5090. The computation of path-traced radiance samples is entirely avoided.
Zoomed comparison showing HaloRT's ability to preserve fine details compared to other real-time methods like OIDN or MRPNN.

Limitations and Future Work

  • Scope: The current model is specialized for cloud-like media. It does not yet handle surfaces (like mountains or buildings) inside the volume, nor does it handle local light sources (like a plane's landing lights) or emissive media (like fire).
  • Approximate Sufficiency: The authors acknowledge that primary-ray features are only approximately sufficient. In very complex lighting scenarios, some global transport effects might be missed.
  • Future Directions: The team aims to extend the feature set to handle smoke and fire, and to integrate temporal recurrence to further improve coherence during fast camera movements.

Summary updated

Abstract

We present a real-time method for reconstructing fully integrated pixel radiance in rendered volumetric media under direct and ambient environmental illumination, using compact, low-cost scene features and statistics. Unlike Monte Carlo denoising pipelines for volumetric rendering, which depend on computationally expensive path-traced radiance estimates, our approach directly infers the final light-transport solution from inexpensive primary-ray and environment sample evaluations. We show that these features form a near-sufficient representation of the scene, enabling accurate reconstruction at substantially lower computational cost. Using an attention-augmented image-to-image network, we demonstrate that complete pixel radiance can be inferred directly from these auxiliary features, without requiring the original radiance signal. Operating in image space also provides a more stable computational cost as the camera zooms into or out of the volume, with substantially less dependence on volume density than approaches based on explicit multiple-scattering simulation. We demonstrate real-time performance and high reconstruction accuracy in challenging volumetric cloud scenarios.
BibTeX
@misc{leonard2026real,
  author = {Ludwic Leonard and Ruediger Westermann},
  title = {{Real-Time Volumetric Light Transport Inference without a Path}},
  year = {2026},
  eprint = {2611.12103},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12103},
}
Figure