Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping

Baorui Ma, Yu-Shen Liu, Zhizhong Han

Introduction

Capturing 3D data is increasingly accessible through LiDAR technology in autonomous vehicles and consumer electronics. However, raw point cloud data is inherently discretized, irregular, and noisy. For high-fidelity applications like Virtual Reality or simulation, this data must be converted into a continuous representation, such as a Signed Distance Function (SDF). Traditional deep learning models for SDFs require clean ground truth or oriented normals, which are often unavailable in real-world scenarios.

Methodology: Noise-to-Noise Mapping in 3D

This paper introduces a self-supervised framework that learns SDFs directly from noisy observations without external supervision. The core idea is a "noise-to-noise" mapping, where the model learns to map one noisy observation ($N_i$) to another ($N_j$) of the same scene.

The proposed framework maps noisy observations to learn an SDF and denoise points simultaneously.

Key Components

  • SDF-Based Denoising: Instead of a separate denoising network, the model uses the learned SDF ($f_\theta$) itself. By utilizing the gradient of the SDF, noisy points are "pulled" toward the estimated surface, a concept inspired by NeuralPull.
  • Earth Mover’s Distance (EMD): To overcome the lack of point-to-point correspondence between different noisy scans, the authors employ EMD. This metric allows the model to perform statistical reasoning across observations by finding the optimal transport between point sets.
  • Geometric Consistency (GC) Regularization: A novel loss term ensures that the predicted SDF values at query points match the actual distances to the denoised point cloud, preventing the model from converging on physically unrealistic surfaces.

Experimental Results

The framework was evaluated on both synthetic benchmarks (PU, PC, ShapeNet) and real-world datasets (D-FAUST, SRB, KITTI).

Performance Benchmarks

  • Denoising: The method achieved state-of-the-art results on the PU dataset with 3% noise, reaching a Chamfer Distance of 4.221, significantly outperforming traditional filters and supervised models.
  • Surface Reconstruction: On ShapeNet, the model surpassed supervised methods in Normal Consistency and F-Score without requiring clean ground truth.
  • Real-World Application: The model demonstrates remarkable robustness on large-scale scans, such as the Paris-rue-Madame dataset, producing smooth surfaces where previous methods often failed.
Visual comparison of surface reconstruction from a single real scan.

Discussion and Future Work

A significant finding is the "Power of One": the model generalizes effectively even from a single noisy scan by performing statistical reasoning across local patches. However, limitations remain regarding extremely high noise levels (above 10%) and the computational cost of calculating EMD for massive point clouds. Future research may extend this noise-to-noise logic to dynamic 3D scenes.

Summary updated

Abstract

Learning signed distance functions (SDFs) from 3D point clouds is an important task in 3D computer vision. However, without ground truth signed distances, point normals or clean point clouds, current methods still struggle from learning SDFs from noisy point clouds. To overcome this challenge, we propose to learn SDFs via a noise to noise mapping, which does not require any clean point cloud or ground truth supervision for training. Our novelty lies in the noise to noise mapping which can infer a highly accurate SDF of a single object or scene from its multiple or even single noisy point cloud observations. Our novel learning manner is supported by modern Lidar systems which capture multiple noisy observations per second. We achieve this by a novel loss which enables statistical reasoning on point clouds and maintains geometric consistency although point clouds are irregular, unordered and have no point correspondence among noisy observations. Our evaluation under the widely used benchmarks demonstrates our superiority over the state-of-the-art methods in surface reconstruction, point cloud denoising and upsampling. Our code, data, and pre-trained models are available at https://github.com/mabaorui/Noise2NoiseMapping/ .
BibTeX
@inproceedings{ma2023learning,
  author = {Baorui Ma and Yu-Shen Liu and Zhizhong Han},
  title = {{Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping}},
  booktitle = {International Conference on Machine Learning},
  year = {2023},
  url = {https://openreview.net/forum?id=2qflscc6A8},
}
Figure