Neighbourhood-Insensitive Point Cloud Normal Estimation Network

Zirui Wang, Victor Prisacariu

Introduction

Surface normal estimation is a foundational task in 3D vision, essential for rendering, reconstruction, and tracking. Traditional methods like Principal Component Analysis (PCA) rely heavily on a hyper-parameter $k$ (the number of nearest neighbors). In complex 3D scenes, a single $k$ value is rarely optimal: flat areas benefit from large neighborhoods to average out noise, while sharp edges or corners require small, localized neighborhoods to avoid "smoothing over" geometry.

Existing deep learning solutions often utilize "Multi-scale" architectures that run several sub-networks on different neighborhood sizes. However, these are computationally expensive, memory-intensive (often >1GB), and slow. This paper introduces a self-attention-based network that uses a learnable temperature parameter to adaptively focus on relevant points, making the model insensitive to the neighborhood range. The result is a model that is 25x smaller and 12x faster than previous state-of-the-art methods while achieving higher accuracy.

Methods: Temperature-adjusted Multi-Head Self-Attention

The authors propose a single-scale network architecture centered around a novel Temperature-adjusted Multi-Head Self-Attention (TMHSA) module.

The proposed network architecture featuring the TMHSA module, which uses a learnable temperature parameter to refine attention weights.

The pipeline consists of four main stages:

  1. Feature Extraction: A local patch of $k$ points is processed through a Multi-Layer Perceptron (MLP) to extract per-point features.
  2. TMHSA Module: Unlike standard self-attention, this module includes a learnable temperature parameter ($t$). This parameter controls the "softness" of the softmax function within the attention mechanism. By learning $t$, the network can decide how "sharply" to focus on specific points within the neighborhood.
  3. Aggregation: A Feed Forward Network (FFN) adds non-linearity, followed by a symmetric max-pooling function that collapses point features into a single global descriptor.
  4. Regression: Finally, fully connected layers regress the 3D unit vector representing the surface normal.

The key innovation is that the model doesn't need a perfectly tuned $k$. With a large neighborhood (e.g., $k=50$), the attention mechanism "softly" ignores points that aren't geometrically relevant to the center point.

Key Results and Insights

The model was evaluated on the PCPNet dataset, a standard benchmark of 27 objects. The results demonstrate significant improvements in both accuracy and efficiency.

Neighbourhood Insensitivity

A striking finding is that the model’s performance does not degrade as the neighborhood size increases. While PCA performance drops significantly if $k$ is too large, this model remains stable or even improves, effectively filtering out irrelevant data.

Comparison of RMS Angle Error and PGP 10 metrics against neighborhood size. The proposed model (Ours) maintains high performance even as $k$ increases, unlike PCA.

Performance Metrics

  • Accuracy: Achieved 94.1% accuracy (PGP 10 metric), outperforming the previous state-of-the-art (91.2%).
  • Efficiency: The model is only 41MB, a 25x reduction compared to Nesti-Net (1.05GB). It processes 100,000 points in 6.5 seconds, a 12x speedup over previous best methods.
  • Downstream Impact: High-quality normals led to significantly faster convergence in Iterative Closest Point (ICP) registration (~11 iterations vs. ~23-28 for other methods).

Conclusion

By replacing brute-force multi-scale sub-networks with a single, smarter attention-based "expert," this work achieves state-of-the-art results with a fraction of the computational cost. While the model is robust to neighborhood variance, future work may explore the impact of extreme sensor noise on the learnable temperature parameter and the integration of global scene context to improve results in sparse regions.

Summary updated

BibTeX
@inproceedings{wang2020neighbourhood,
  author = {Zirui Wang and Victor Prisacariu},
  title = {{Neighbourhood-Insensitive Point Cloud Normal Estimation Network}},
  booktitle = {BMVC},
  year = {2020},
  url = {https://openreview.net/forum?id=x8CuhO6MAyR},
}
Figure