3D Point Cloud Denoising via Deep Neural Network Based Local Surface Estimation

Chaojing Duan, Siheng Chen, Jelena Kovacevic

TL;DR

This paper introduces Neural Projection Denoising (NPD), a novel deep learning approach for cleaning noisy 3D point clouds. By using a neural network to accurately estimate local surface planes for each point, NPD achieves superior denoising performance and stability compared to existing methods, all with a single projection step.

What Problem Does This Paper Address?

3D point clouds, generated by sensors or reconstruction techniques, are often corrupted by noise. This noise can significantly degrade the performance of downstream tasks like surface reconstruction and geometric processing, as it distorts the underlying shape and loses fine details. Traditional denoising methods struggle with the inherent permutation invariance of point clouds and the difficulty of adaptively defining local neighborhoods, especially in areas with varying sampling densities or sharp geometric features. The authors aim to overcome these limitations by leveraging deep learning to directly estimate the local surface geometry for each point.

Methods

The core of the NPD algorithm is a neural network architecture designed to estimate local reference planes for each point in a noisy 3D point cloud. The network takes the noisy point cloud as input and processes it through shared Multi-Layer Perceptrons (MLPs) to extract local features. Global features are captured via max-pooling across all points. These local and global features are then concatenated and fed through further MLPs to predict an N x 4 matrix representing the reference planes (normal vector and interception) for each of the N points.

Once these reference planes are estimated, each noisy point is projected onto its corresponding estimated plane. This projection step effectively denoises the point cloud. The training process involves two loss functions: loss1 (Mean Squared Error) to ensure the denoised points are close to the ground-truth noiseless points, and loss2 (cosine similarity loss) to ensure the estimated reference planes are similar to the ground-truth reference planes. A weighted combination of these losses is used for training, with a parameter α controlling the balance between point-wise accuracy and geometric plane accuracy.

Datasets

  • Training: ShapeNet dataset, containing 55 object categories and over 50,000 3D mesh models. 40,000 meshes were used, with point clouds sampled to contain 2048 points each.
  • Testing: ModelNet10 dataset, comprising 10 object categories and over 3000 3D mesh models. 50 meshes were randomly selected from seven categories, and point clouds were sampled for testing.

Key Insights and Ideas

  • Deep Learning for Local Geometry: The paper demonstrates that deep neural networks can effectively learn to estimate local surface geometry (reference planes) directly from noisy point clouds, a task traditionally handled by more heuristic geometric methods.
  • Single-Pass Denoising: By accurately estimating reference planes, NPD achieves effective denoising with a single projection step, contrasting with previous multi-projection methods that were more computationally intensive and sensitive to noise in the estimated planes.
  • Global and Local Feature Fusion: The architecture's success hinges on effectively combining local point features with global scene context. This fusion makes the algorithm more robust to variations in sampling density and geometric complexity.
  • Adaptive Neighborhoods Implicitly Learned: The neural network implicitly learns to adapt to local neighborhoods without explicit neighborhood definition or eigendecomposition, overcoming a major challenge in point cloud processing.

Key Results

  • NPD outperforms four out of five competing denoising algorithms (BF, PDE, GBD, NLD) across seven categories on both MSE and Chamfer Distance metrics.
  • The proposed algorithm consistently produces significantly smaller variances in its denoising results compared to other methods, indicating greater stability and reliability.
  • The inclusion of global features in the network architecture significantly improves denoising performance, as shown by a comparison in Table 1.

Limitations and Future Work

  • Parameter Tuning (α): The paper notes that the parameter α, which balances point-to-plane and plane-to-plane losses, needs to be tuned for optimal performance. The authors suggest a strategy of starting with small α and gradually increasing it during training.
  • Computational Cost: While NPD offers a single-pass projection, the deep learning architecture itself can be computationally intensive during training.
  • Future Work: The authors suggest that NPD provides a foundation for 3D point cloud parameterization by combining local and global information, hinting at potential applications beyond denoising. The code is made publicly available, encouraging further research and development.

Summary updated

Abstract

We present a neural-network-based architecture for 3D point cloud denoising called neural projection denoising (NPD). In our previous work, we proposed a two-stage denoising algorithm, which first estimates reference planes and follows by projecting noisy points to estimated reference planes. Since the estimated reference planes are inevitably noisy, multi-projection is applied to stabilize the denoising performance. NPD algorithm uses a neural network to estimate reference planes for points in noisy point clouds. With more accurate estimations of reference planes, we are able to achieve better denoising performances with only one-time projection. To the best of our knowledge, NPD is the first work to denoise 3D point clouds with deep learning techniques. To conduct the experiments, we sample 40000 point clouds from the 3D data in ShapeNet to train a network and sample 350 point clouds from the 3D data in ModelNet10 to test. Experimental results show that our algorithm can estimate normal vectors of points in noisy point clouds. Comparing to five competitive methods, the proposed algorithm achieves better denoising performance and produces much smaller variances. Our code is available at https://github.com/chaojingduan/Neural-Projection.
BibTeX
@inproceedings{duan2019d,
  author = {Chaojing Duan and Siheng Chen and Jelena Kovacevic},
  title = {{3D Point Cloud Denoising via Deep Neural Network Based Local Surface Estimation}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=RTSuRGvmFRm},
}
Figure