Neural Poisson: Indicator Functions for Neural Fields

Angela Dai, Matthias NießnerarXiv 2022 · DOI · arXiv

TL;DR

"Neural Poisson" shifts the paradigm of 3D surface reconstruction by representing shapes as indicator functions (0 for empty space, 1 for solid) rather than traditional Signed Distance Fields (SDFs). This approach allows for the direct integration of "empty space" observations—the clear line-of-sight between a sensor and an object—into the neural network's loss function. The result is a significantly cleaner reconstruction with a 9.5% improvement in accuracy over previous state-of-the-art methods.

Addressing the "Ballooning" Problem in 3D Reconstruction

When scanning environments with LiDAR or mobile depth sensors, we obtain more than just surface points; we also gain information about where geometry cannot exist. Specifically, the space between the sensor and the detected surface must be empty.

Traditional neural reconstruction methods typically rely on Signed Distance Fields (SDFs). In an SDF, every point in space stores its distance to the nearest surface. However, SDFs are poorly suited for handling "empty space" data because distance values in a void change continuously as the sensor moves. This often results in "ballooning" artifacts—spurious surfaces appearing in unobserved or occluded regions—because the model lacks a robust mathematical framework to enforce emptiness.

Methods: From Poisson to Neural Fields

The authors propose a neural field formulation inspired by classical Poisson Surface Reconstruction, adapted for deep learning.

The Neural Poisson pipeline utilizes an MLP to predict an indicator function, supervised by Gradient, Surface, and Empty Space losses.
  1. Indicator Functions: Instead of predicting distance, the Multilayer Perceptron (MLP) predicts an indicator function $\chi(x)$, where $\chi=0$ represents space outside the object and $\chi=1$ represents the interior.
  2. The Gradient Constraint: Aligning with Poisson reconstruction principles, the gradient of the indicator function is constrained to match surface normals. A "Gradient Loss" ensures the MLP's output transitions sharply and accurately at the boundary.
  3. Empty Space Loss: By using an indicator function, "empty" becomes a constant value (0). The authors sample points along the camera's line-of-sight and penalize the network if it predicts non-zero values in these regions.
  4. Surface Loss: The MLP is constrained to output exactly $0.5$ at observed surface points, pinning the geometric boundary to the input data.
  5. Architecture: The model employs a 5-layer MLP with sine activations (SIREN), which excels at capturing high-frequency geometric details like fabric wrinkles or thin structures.

Datasets and Evaluation

The approach was evaluated on two distinct datasets:

  • 3D-FRONT: A large-scale synthetic dataset of furnished indoor rooms, providing "ground truth" for precise quantitative measurement of reconstruction error.
  • ARKitScenes: A real-world dataset of indoor scans from mobile devices, testing the model's robustness against the noise and incompleteness of consumer-grade hardware.

Key Results and Insights

The shift to indicator functions provides several critical advantages:

  • Quantitative Superiority: The method achieved a 9.5% improvement in Chamfer Distance over previous leaders like Siren and traditional Poisson methods.
  • Qualitative Cleanliness: Neural Poisson effectively eliminates "floaters" and "ballooning" in occluded areas, such as the space behind cabinets or under furniture, where SDF-based methods often fail.
  • Detail Retention: The model remains sharp enough to reconstruct fine-grained details, such as bed sheet wrinkles, that are frequently smoothed away by competing algorithms.
Qualitative comparison of Neural Poisson against state-of-the-art methods (Siren, SSD, SPSR) across various indoor scenes.

Limitations and Future Work

While highly effective, the "Empty Space Constraint" depends on accurate sensor poses. Noisy or drifting camera tracking can lead to conflicting "empty" rays that degrade the reconstruction. Additionally, unobserved regions (e.g., the back of a chair against a wall) remain a challenge. Future research may involve incorporating data-driven priors to "hallucinate" these unobserved surfaces or jointly optimizing camera paths and surfaces to correct tracking errors in real-time.

Summary updated

Abstract

Implicit neural field generating signed distance field representations (SDFs) of 3D shapes have shown remarkable progress in 3D shape reconstruction and generation. We introduce a new paradigm for neural field representations of 3D scenes; rather than characterizing surfaces as SDFs, we propose a Poisson-inspired characterization for surfaces as indicator functions optimized by neural fields. Crucially, for reconstruction of real scan data, the indicator function representation enables simple and effective constraints based on common range sensing inputs, which indicate empty space based on line of sight. Such empty space information is intrinsic to the scanning process, and incorporating this knowledge enables more accurate surface reconstruction. We show that our approach demonstrates state-of-the-art reconstruction performance on both synthetic and real scanned 3D scene data, with 9.5% improvement in Chamfer distance over state of the art.
BibTeX
@misc{dai2022neural,
  author = {Angela Dai and Matthias Nießner},
  title = {{Neural Poisson: Indicator Functions for Neural Fields}},
  year = {2022},
  eprint = {2211.14249},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2211.14249},
}
Figure