Abstract
Deep normal estimators have made great strides on synthetic benchmarks. Unfortunately, their performance dramatically drops on real scan data since they are typically supervised only on synthetic datasets. The point-wise annotation of ground truth normals is vulnerable to inefficiency and inaccuracies, making it nearly impossible to build perfect real datasets for supervised deep learning. To overcome this challenge, we propose a multi-sample consensus paradigm for unsupervised normal estimation. This paradigm consists of multi-candidate sampling, candidate rejection, and mode determination. We introduce PointNorm-Net (MSUNE-Net), the first unsupervised deep normal estimator. It transfers online consensus stages to offline training, resulting in inference times 100 times faster than optimization-based methods while achieving superior accuracy. Comprehensive experiments demonstrate that our unsupervised methods outperform state-of-the-art supervised estimators on both synthetic and real-world datasets like NYUV2 and KITTI.
Introduction
Surface normals—vectors perpendicular to a surface at a given point—are fundamental for 3D tasks such as robot grasping, autonomous driving, and scene reconstruction. While deep learning has revolutionized this field, most existing models are supervised, requiring massive amounts of labeled data. This approach faces two major hurdles: the annotation gap, where manual labeling of millions of points is impractical, and the domain gap, where real-world sensor noise (LiDAR, Kinect) differs significantly from synthetic training noise. PointNorm-Net addresses these by learning directly from raw, unlabeled point clouds, ensuring robustness to specific sensor characteristics.
Methodology
The core of PointNorm-Net is a three-stage multi-modal distribution estimation paradigm. It leverages a mathematical property established by the authors: on a smooth surface with zero-mean noise, the ground-truth normal is the expectation of randomly sampled candidate normals.
The pipeline consists of:
- Candidate Normal Initialization (CNI): Randomly sampling neighboring point groups to fit local planes and generate candidate normals. The model uses adaptive neighborhoods to adjust to varying noise levels.
- Feasible Candidate Filtering (FCF): Using kernel density estimation to filter out "blurry" normals that lack strong consensus among neighboring points.
- Major Mode Estimation: Employing a novel Candidate Consensus Loss ($L_{ccn}$) to train a neural network to identify the most representative normal in multi-modal distributions (e.g., near sharp edges). A secondary feature-preserving loss ($L_w$) ensures that points on different sides of an edge do not interfere with each other, preventing over-smoothing.
Experimental Results
PointNorm-Net was evaluated across a diverse range of data sources, including real-world Kinect (PCV, NYUV2), LiDAR (KITTI), and TLS (Semantic3D) datasets.
Key findings include:
- Superior Generalization: On the PCV Kinect dataset, PointNorm-Net achieved an RMS error of 10.89, significantly lower than the supervised DeepFit (13.68) and traditional optimization methods like HF-cubes (11.72).
- Efficiency: The model requires only a single forward pass (0.36ms per point), matching the speed of supervised models while being significantly faster than traditional optimization-based estimators.
- Robustness: High accuracy was maintained across different sensor types without per-sensor retraining, demonstrating its ability to bridge the domain gap effectively.
Conclusion
PointNorm-Net represents a significant step forward as the first self-supervised deep learning framework for normal estimation. By framing the problem as finding the major mode of a multi-modal distribution, it preserves sharp geometric features while remaining robust to real-world noise. While challenges remain regarding extremely thin structures and non-uniform point densities, the multi-sample consensus framework offers a promising path for unsupervised 3D scene understanding.
Summary updated