Learnable Skeleton-Aware 3D Point Cloud Sampling

Cheng Wen, Baosheng Yu, Dacheng Tao

Abstract

Efficiently processing 3D point clouds is a critical challenge in computer vision, particularly for real-time applications. Traditional downsampling methods often overlook the intrinsic geometric structure of objects, leading to the loss of fine details. This paper presents a novel skeleton-aware sampling approach that utilizes the medial axis of an object as a structural prior. By integrating unsupervised skeleton learning with a differentiable sampling pipeline, the proposed method identifies and preserves points essential to an object's topology. Experimental results on the ModelNet40 dataset demonstrate that this approach achieves state-of-the-art performance in classification and reconstruction tasks while significantly reducing data volume.

Introduction

The proliferation of 3D sensors, such as LiDAR and depth cameras, has led to an explosion of point cloud data. While rich in information, these massive datasets are computationally expensive to process, especially on mobile or edge devices. Consequently, point cloud sampling—selecting a representative subset of points—has become a fundamental preprocessing step.

Existing sampling techniques like Random Sampling (RS) or Farthest Point Sampling (FPS) are widely used due to their simplicity but suffer from being "geometry-blind." They often discard critical structural features, such as thin limbs or sharp edges, because they lack an understanding of the object's underlying shape. While recent "learn-to-sample" methods attempt to optimize sampling for specific downstream tasks, they frequently ignore the object's intrinsic topology. This paper addresses these limitations by introducing a sampling method guided by the object's skeleton.

Overview of the skeleton-aware sampling process: (a) input point cloud, (b) learned skeleton, (c-d) sampling results.

Methodology

The proposed framework consists of a two-stage pipeline designed to learn and utilize structural priors for sampling.

Unsupervised Skeleton Estimation

A core innovation of this work is the unsupervised learning of the "medial axis"—the internal skeleton of a 3D shape. Rather than relying on human-labeled data, the model predicts a set of "skeletal spheres" defined by their centers and radii. To accurately locate these centers, the authors employ Delaunay neighbors instead of standard K-Nearest Neighbors (KNN). This choice allows the model to better capture "antipodal" points—points on opposite sides of a structure—which are essential for identifying the true geometric center.

Local Feature Size (LFS) Prior

The skeleton serves as the basis for calculating the Local Feature Size (LFS). The LFS at any surface point is defined as the distance to the nearest point on the medial axis. This metric acts as a proxy for geometric importance: points with a small LFS typically reside in high-detail or delicate areas (like edges and thin structures), whereas points with a large LFS are found on broader, flatter surfaces.

The medial axis and Local Feature Size (LFS) provide a geometric blueprint for identifying important surface points.

Differentiable Sampling Pipeline

To enable end-to-end training, the sampling process must be differentiable. However, selecting a discrete subset of points is inherently non-differentiable. The authors overcome this by using the Gumbel-softmax reparameterization trick. This technique transforms the "hard" discrete choice into a "soft" probability distribution during training, allowing the network to optimize the sampler based on its performance in downstream tasks, such as object classification.

Experimental Evaluation

The method was evaluated using the ModelNet40 benchmark, which contains over 12,000 CAD models across 40 categories.

Key Results

  • Classification Accuracy: When downsampling to just 128 points (an 8x reduction from the standard 1024), the method achieved an accuracy of 89.1%. This performance is nearly identical to the 89.5% accuracy achieved using the full point cloud, significantly outperforming other learnable samplers like S-NET and SampleNet.
  • Shape Reconstruction: The method achieved the lowest Chamfer Distance in reconstruction tasks, indicating that the sampled points preserve the original geometry more accurately than competing methods.
  • Object Retrieval: The sampled points retained the unique "identity" of shapes better than random or distance-based sampling, leading to superior performance in retrieval benchmarks.

Discussion and Conclusion

By treating the skeleton as a structural blueprint, this research demonstrates that geometric priors can significantly enhance the efficiency and accuracy of 3D point cloud processing. The use of LFS as an importance map allows the model to make informed sampling decisions even before task-specific learning begins.

Despite its success, the method faces challenges regarding computational complexity, as predicting a high number of skeletal spheres increases FLOPs. Additionally, like many skeleton-based approaches, it remains sensitive to sensor noise. Future work will focus on scaling this approach to complex, large-scale scenes and improving robustness against noisy input data. In conclusion, skeleton-aware sampling represents a significant step forward in bridging the gap between raw 3D data and efficient geometric understanding.

Summary updated

Abstract

Point cloud sampling is crucial for efficient large-scale point cloud analysis, where learning-to-sample methods have recently received increasing attention from the community for jointly training with downstream tasks. However, the abovementioned task-specific sampling methods usually fail to explore the geometries of objects in an explicit manner. In this paper, we introduce a new skeleton-aware learning-to-sample method by learning object skeletons as the prior knowledge to preserve the object geometry and topology information during sampling. Specifically, without labor-intensive annotations per object category, we first learn category-agnostic object skeletons via the medial axis transform definition in an unsupervised manner. With object skeleton, we then evaluate the histogram of the local feature size as the prior knowledge to formulate skeleton-aware sampling from a probabilistic perspective. Additionally, the proposed skeleton-aware sampling pipeline with the task network is thus end-to-end trainable by exploring the reparameterization trick. Extensive experiments on three popular downstream tasks, point cloud classification, retrieval, and reconstruction, demonstrate the effectiveness of the proposed method for efficient point cloud analysis.
BibTeX
@inproceedings{wen2023learnable,
  author = {Cheng Wen and Baosheng Yu and Dacheng Tao},
  title = {{Learnable Skeleton-Aware 3D Point Cloud Sampling}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.01695},
}
Figure