TL;DR
"Neural Poisson" shifts the paradigm of 3D surface reconstruction by representing shapes as indicator functions (0 for empty space, 1 for solid) rather than traditional Signed Distance Fields (SDFs). This approach allows for the direct integration of "empty space" observations—the clear line-of-sight between a sensor and an object—into the neural network's loss function. The result is a significantly cleaner reconstruction with a 9.5% improvement in accuracy over previous state-of-the-art methods.
Addressing the "Ballooning" Problem in 3D Reconstruction
When scanning environments with LiDAR or mobile depth sensors, we obtain more than just surface points; we also gain information about where geometry cannot exist. Specifically, the space between the sensor and the detected surface must be empty.
Traditional neural reconstruction methods typically rely on Signed Distance Fields (SDFs). In an SDF, every point in space stores its distance to the nearest surface. However, SDFs are poorly suited for handling "empty space" data because distance values in a void change continuously as the sensor moves. This often results in "ballooning" artifacts—spurious surfaces appearing in unobserved or occluded regions—because the model lacks a robust mathematical framework to enforce emptiness.
Methods: From Poisson to Neural Fields
The authors propose a neural field formulation inspired by classical Poisson Surface Reconstruction, adapted for deep learning.
- Indicator Functions: Instead of predicting distance, the Multilayer Perceptron (MLP) predicts an indicator function $\chi(x)$, where $\chi=0$ represents space outside the object and $\chi=1$ represents the interior.
- The Gradient Constraint: Aligning with Poisson reconstruction principles, the gradient of the indicator function is constrained to match surface normals. A "Gradient Loss" ensures the MLP's output transitions sharply and accurately at the boundary.
- Empty Space Loss: By using an indicator function, "empty" becomes a constant value (0). The authors sample points along the camera's line-of-sight and penalize the network if it predicts non-zero values in these regions.
- Surface Loss: The MLP is constrained to output exactly $0.5$ at observed surface points, pinning the geometric boundary to the input data.
- Architecture: The model employs a 5-layer MLP with sine activations (SIREN), which excels at capturing high-frequency geometric details like fabric wrinkles or thin structures.
Datasets and Evaluation
The approach was evaluated on two distinct datasets:
- 3D-FRONT: A large-scale synthetic dataset of furnished indoor rooms, providing "ground truth" for precise quantitative measurement of reconstruction error.
- ARKitScenes: A real-world dataset of indoor scans from mobile devices, testing the model's robustness against the noise and incompleteness of consumer-grade hardware.
Key Results and Insights
The shift to indicator functions provides several critical advantages:
- Quantitative Superiority: The method achieved a 9.5% improvement in Chamfer Distance over previous leaders like Siren and traditional Poisson methods.
- Qualitative Cleanliness: Neural Poisson effectively eliminates "floaters" and "ballooning" in occluded areas, such as the space behind cabinets or under furniture, where SDF-based methods often fail.
- Detail Retention: The model remains sharp enough to reconstruct fine-grained details, such as bed sheet wrinkles, that are frequently smoothed away by competing algorithms.
Limitations and Future Work
While highly effective, the "Empty Space Constraint" depends on accurate sensor poses. Noisy or drifting camera tracking can lead to conflicting "empty" rays that degrade the reconstruction. Additionally, unobserved regions (e.g., the back of a chair against a wall) remain a challenge. Future research may involve incorporating data-driven priors to "hallucinate" these unobserved surfaces or jointly optimizing camera paths and surfaces to correct tracking errors in real-time.
Summary updated