Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping

Baorui Ma, Yu-Shen Liu, Zhizhong Han

Introduction

Capturing 3D data is increasingly accessible through LiDAR technology in autonomous vehicles and consumer electronics. However, raw point cloud data is inherently discretized, irregular, and noisy. For high-fidelity applications like Virtual Reality or simulation, this data must be converted into a continuous representation, such as a Signed Distance Function (SDF). Traditional deep learning models for SDFs require clean ground truth or oriented normals, which are often unavailable in real-world scenarios.

Methodology: Noise-to-Noise Mapping in 3D

This paper introduces a self-supervised framework that learns SDFs directly from noisy observations without external supervision. The core idea is a "noise-to-noise" mapping, where the model learns to map one noisy observation ($N_i$) to another ($N_j$) of the same scene.

The proposed framework maps noisy observations to learn an SDF and denoise points simultaneously.

Key Components

  • SDF-Based Denoising: Instead of a separate denoising network, the model uses the learned SDF ($f_\theta$) itself. By utilizing the gradient of the SDF, noisy points are "pulled" toward the estimated surface, a concept inspired by NeuralPull.
  • Earth Mover’s Distance (EMD): To overcome the lack of point-to-point correspondence between different noisy scans, the authors employ EMD. This metric allows the model to perform statistical reasoning across observations by finding the optimal transport between point sets.
  • Geometric Consistency (GC) Regularization: A novel loss term ensures that the predicted SDF values at query points match the actual distances to the denoised point cloud, preventing the model from converging on physically unrealistic surfaces.

Experimental Results

The framework was evaluated on both synthetic benchmarks (PU, PC, ShapeNet) and real-world datasets (D-FAUST, SRB, KITTI).

Performance Benchmarks

  • Denoising: The method achieved state-of-the-art results on the PU dataset with 3% noise, reaching a Chamfer Distance of 4.221, significantly outperforming traditional filters and supervised models.
  • Surface Reconstruction: On ShapeNet, the model surpassed supervised methods in Normal Consistency and F-Score without requiring clean ground truth.
  • Real-World Application: The model demonstrates remarkable robustness on large-scale scans, such as the Paris-rue-Madame dataset, producing smooth surfaces where previous methods often failed.
Visual comparison of surface reconstruction from a single real scan.

Discussion and Future Work

A significant finding is the "Power of One": the model generalizes effectively even from a single noisy scan by performing statistical reasoning across local patches. However, limitations remain regarding extremely high noise levels (above 10%) and the computational cost of calculating EMD for massive point clouds. Future research may extend this noise-to-noise logic to dynamic 3D scenes.

Summary updated

BibTeX
@inproceedings{ma2023learning,
  author = {Baorui Ma and Yu-Shen Liu and Zhizhong Han},
  title = {{Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping}},
  booktitle = {International Conference on Machine Learning},
  year = {2023},
  url = {https://openreview.net/forum?id=2qflscc6A8},
}
Figure