ProbeSDF: Light Field Probes For Neural Surface Reconstruction

Briac Toussaint, Diego Thomas, Jean-Sébastien FrancoCVPR 2025 · DOI

Abstract

ProbeSDF introduces a highly efficient framework for reconstructing 3D surfaces from images by decoupling spatial details from angular lighting information. By storing view-dependent appearance in a low-resolution "probe grid" and geometry in a high-resolution voxel grid, the method achieves state-of-the-art reconstruction quality with training times measured in minutes rather than hours. This architecture supports real-time rendering and handles complex specularities through reflected vector lookups and a tiny neural decoder.

Introduction

Neural surface reconstruction—the process of synthesizing high-fidelity 3D models from 2D image sets—has advanced significantly with methods like NeuS and Neuralangelo. However, these approaches often struggle with a fundamental "speed vs. quality" trade-off. The primary bottleneck is the representation of the Light Field, a 5D function describing the color of a point from a specific viewing direction.

Most current methods co-locate spatial features (geometry) and angular features (appearance) within the same high-resolution data structures. This is computationally redundant because, while surface geometry and texture vary rapidly, environmental lighting typically changes much more slowly across space. ProbeSDF addresses this by minimally reformulating the appearance model to decouple these components, as shown in the pipeline overview below.

Figure 1: The ProbeSDF pipeline decouples low-resolution neural probes for angular features from a high-resolution voxel grid for spatial features, enabling fast training and high-quality reconstruction.

Methodology

ProbeSDF's efficiency stems from its physically inspired decoupled parametrization. The model utilizes two distinct volumetric grids:

  1. High-Resolution Voxel Grid: This grid stores spatial features ($F_s$) and Signed Distance Function (SDF) values to define the precise geometry of the scene.
  2. Low-Resolution Probe Grid: Angular features ($F_a$) are stored using Spherical Harmonics in a grid placed at 1/16th the resolution of the spatial grid. These "probes" capture the smooth spatial variation of environmental lighting.

To handle shiny or specular surfaces, the model samples the angular grid using the reflected ray direction ($r$) rather than the view direction alone. The final color prediction is performed by a "tiny" Multi-Layer Perceptron (MLP) with only two hidden layers of 32 neurons. This minimal decoder is sufficient because the input features are already highly organized and physically grounded. Additionally, a polynomial term is included to approximate the Fresnel effect, accounting for increased reflectivity at grazing angles.

Figure 2: Detailed architecture showing the 16x16x16 probe grid, reflected vector calculation, and the fusion of spatial and angular features into a tiny MLP for RGB prediction.

Key Insights and Implementation

The core insight of ProbeSDF is that lighting is "locally invariant." By encoding environment components at a lower spatial resolution, the model significantly reduces memory and compute requirements without sacrificing visual fidelity. To maximize performance, the entire computation—from grid lookups to the MLP inference—is fused into a single CUDA kernel, enabling real-time rendering speeds of 200Hz to 400Hz.

Experimental Results

The authors evaluated ProbeSDF across several challenging datasets, including MVMannequins, ActorsHQ (4K video), DTU, and BlendedMVS.

  • Training Efficiency: On the MVMannequins dataset, ProbeSDF converges in approximately 1 minute, significantly faster than NeuS2 (5 minutes) and Voxurf (15 minutes).
  • Reconstruction Accuracy: On the DTU benchmark, it achieved a Chamfer Distance of 0.68mm, outperforming Voxurf (0.73mm) and 2D Gaussian Splatting (0.76mm).
  • High-Resolution Performance: The model scales gracefully to 4K datasets like ActorsHQ, producing sharper details on clothing and skin where other methods often require image downsampling to remain computationally feasible.
  • Compactness: The resulting models are highly portable, with a typical scene footprint of around 30MB.

Conclusion and Future Work

ProbeSDF represents a significant step toward practical, real-time neural 3D reconstruction. By leveraging the spatial smoothness of lighting through a probe-based architecture, it breaks the traditional speed-quality barrier. Future research may focus on further disentangling these probes into explicit light sources and material properties (BRDFs) to enable full scene relighting, as well as improving extrapolation in areas far from the original camera viewpoints.

Summary updated

BibTeX
@inproceedings{toussaint2025probesdf,
  author = {Briac Toussaint and Diego Thomas and Jean-Sébastien Franco},
  title = {{ProbeSDF: Light Field Probes For Neural Surface Reconstruction}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.01030},
}
Figure