A comprehensive review of 3D point cloud descriptors.

Xian-Feng Han, Jesse S. Jin, Xie Juan, Mingjie Wang, Wei JiangarXiv 2018 · arXiv

Abstract

The introduction of inexpensive 3D data acquisition devices has promisingly facilitated the wide availability and popularity of 3D point cloud, which attracts more attention on the effective extraction of novel 3D point cloud descriptors for accurate and efficient of 3D computer vision tasks. However, how to develop discriminative and robust feature descriptors from various point clouds remains a challenging task. This paper comprehensively investigates the existing approaches for extracting 3D point cloud descriptors which are categorized into three major classes: local-based descriptor, global-based descriptor and hybrid-based descriptor. Furthermore, experiments are carried out to present a thorough evaluation of performance of several state-of-the-art 3D point cloud descriptors used widely in practice in terms of descriptiveness, robustness and efficiency.

Introduction

3D point clouds, represented as sets of $(x, y, z)$ coordinates, serve as the fundamental data structure for spatial sensing technologies like LiDAR and RGB-D cameras. Unlike structured 2D images, point clouds are inherently sparse, irregular, and unordered. The primary challenge in 3D computer vision is "descriptiveness": transforming these raw coordinates into robust feature vectors (descriptors) that remain invariant to noise, rotation, and scale. This paper provides a systematic review of the evolution of these descriptors, bridging the gap between traditional geometric engineering and modern deep learning.

Taxonomy of 3D Point Cloud Descriptors

To organize the vast landscape of feature extraction methods, we propose a unified taxonomy that distinguishes between traditional hand-crafted techniques and contemporary data-driven models.

Taxonomy of 3D point cloud descriptors, categorized into hand-crafted and deep learning approaches.

The Hand-crafted Age

Traditional methods rely on fixed mathematical formulations to capture geometric properties:

  • Local Descriptors: These focus on neighborhoods around keypoints, often utilizing a Local Reference Frame (LRF) to ensure rotation invariance (e.g., SHOT, ISS).
  • Global Descriptors: These characterize the entire object geometry, making them suitable for rapid object retrieval (e.g., VFH, ESF).
  • Hybrid Descriptors: These attempt to combine local details with global context to improve discriminative power.

The Deep Learning Age

Modern architectures learn features directly from data, categorized by their input representation:

  • Voxel-based: Points are quantized into a 3D grid. While compatible with standard CNNs, this approach is memory-intensive and often loses fine-grained detail.
  • Point-based: Pioneered by PointNet, these models process raw coordinates directly using permutation-invariant functions like max-pooling.
  • Graph and Tree-based: These capture local relationships by treating points as nodes in a graph (e.g., DGCNN) or organizing them into hierarchical structures like Octrees or Kd-trees.

Datasets and Benchmarks

The evaluation of 3D descriptors relies on diverse benchmarks spanning object recognition, part segmentation, and large-scale scene understanding.

Overview of standard 3D datasets used for recognition, segmentation, and registration tasks.

Key benchmarks include ModelNet40 for synthetic object recognition, where deep learning methods like SO-Net (93.4%) and PointNet++ (91.9%) have set new performance standards. For real-world indoor scenes, the S3DIS dataset is frequently used to evaluate semantic segmentation, highlighting the strength of graph-based methods in capturing local context compared to early point-based models.

Key Insights and Future Directions

The field has shifted significantly toward processing raw points, realizing that "permutation invariance" is key to handling unordered 3D data. However, several challenges remain:

  1. Scalability: Processing millions of points in city-scale scenes (e.g., SemanticKITTI) remains a major engineering hurdle.
  2. Robustness: Bridging the gap between clean synthetic data and noisy, occluded real-world sensor data.
  3. Multimodal Fusion: Effectively integrating 2D color/texture information with 3D geometry for a unified "RGB-D" experience.
  4. Unsupervised Learning: Reducing the reliance on massive manually labeled datasets through self-supervised feature extraction.

Conclusion

The transition from hand-crafted geometry to deep learning has revolutionized 3D point cloud analysis. While point-based and graph-based architectures currently lead in performance, the future lies in developing more efficient, robust, and multimodal descriptors capable of handling the complexities of real-world environments.

Summary updated

Abstract

The introduction of inexpensive 3D data acquisition devices has promisingly facilitated the wide availability and popularity of 3D point cloud, which attracts more attention on the effective extraction of novel 3D point cloud descriptors for accurate and efficient of 3D computer vision tasks. However, how to de- velop discriminative and robust feature descriptors from various point clouds remains a challenging task. This paper comprehensively investigates the exist- ing approaches for extracting 3D point cloud descriptors which are categorized into three major classes: local-based descriptor, global-based descriptor and hybrid-based descriptor. Furthermore, experiments are carried out to present a thorough evaluation of performance of several state-of-the-art 3D point cloud descriptors used widely in practice in terms of descriptiveness, robustness and efficiency.
BibTeX
@misc{han2018comprehensive,
  author = {Xian-Feng Han and Jesse S. Jin and Xie Juan and Mingjie Wang and Wei Jiang},
  title = {{A comprehensive review of 3D point cloud descriptors.}},
  year = {2018},
  eprint = {1802.02297},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1802.02297},
}
Figure