Abstract
The increased availability of massive point clouds coupled with their utility in a wide variety of applications such as robotics, shape synthesis, and self-driving cars has attracted increased attention from both industry and academia. Recently, deep neural networks operating on labeled point clouds have shown promising results on supervised learning tasks like classification and segmentation. However, supervised learning leads to the cumbersome task of annotating the point clouds. To combat this problem, we propose two novel self-supervised pre-training tasks that encode a hierarchical partitioning of the point clouds using a cover-tree, where point cloud subsets lie within balls of varying radii at each level of the cover-tree. Furthermore, our self-supervised learning network is restricted to pre-train on the support set (comprising of scarce training examples) used to train the downstream network in a few-shot learning (FSL) setting. Finally, the fully-trained self-supervised network's point embeddings are input to the downstream task's network. We present a comprehensive empirical evaluation of our method on both downstream classification and segmentation tasks and show that supervised methods pre-trained with our self-supervised learning method significantly improve the accuracy of state-of-the-art methods. Additionally, our method also outperforms previous unsupervised methods in downstream classification tasks.
Introduction
Deep learning on 3D point clouds typically requires massive amounts of manually annotated data, which is expensive and time-consuming to gather. While unsupervised and self-supervised methods exist, they often suffer from significant drawbacks: they either convert irregular point clouds into memory-intensive voxel grids or rely on computationally heavy reconstruction metrics like Chamfer distance or Earth Mover's Distance.
Applying deep learning to point clouds in a few-shot learning (FSL) scenario—where the model must learn to classify or segment objects given only a handful of examples—is particularly challenging. This research bridges this gap by designing a self-supervised pretext task that operates directly on raw point clouds, avoids expensive distance metrics, and drastically improves sample efficiency for downstream FSL tasks.
Methodology
To avoid voxelization while capturing the multi-scale structure of a point cloud, the method utilizes a metric space indexing data structure called a cover-tree. This approach involves two primary stages:
1. Hierarchical Partitioning
The cover-tree groups the raw point cloud into overlapping "balls" of varying radii. At the top of the tree, a few large balls cover the whole object. As one moves down the tree, the balls become smaller and more numerous, capturing finer local details.
2. Self-Supervised Pretext Tasks
Surrogate labels are generated directly from the cover-tree structure to train a feature extractor. Two parallel tasks are designed to teach the network spatial relationships:
- Task 1 (Regression / Inter-ball): The network is given the centers of two balls at the same level of the tree and must predict the $L_2$ (Euclidean) distance between them. This teaches global spatial relationships.
- Task 2 (Classification / Intra-ball): The network examines a parent ball and one of its child balls (spanning consecutive tree levels) and must classify which of four spatial quadrants the child ball falls into. This teaches local spatial relationships.
3. Downstream Fine-tuning
Once the feature extractor (e.g., PointNet or DGCNN) is pre-trained on these pretext tasks using only the scarce FSL support set, its embeddings are used to initialize the network for the actual downstream task, such as classification or segmentation.
Experimental Results
The method was evaluated across several standard 3D benchmarks adapted for the few-shot learning (m-shot, K-way) setting:
- 3D Object Classification: On the sparse Sydney Urban Objects dataset (10-way 20-shot), a DGCNN initialized with this method achieved 81.50% accuracy, significantly outperforming the random initialization baseline (76.10%) and the previous state-of-the-art VoxelSSL (69.10%).
- Part Segmentation: On ShapeNet (10-way 20-shot), the pre-trained DGCNN achieved 36.91% mean IoU, a substantial improvement over the VoxelNet baseline (26.29%).
- Semantic Segmentation: On S3DIS (5-way 10-shot), the method achieved 68.87% accuracy, improving upon the VoxelSSL baseline (64.59%).
Qualitative Analysis and Insights
A key finding of this work is that semantic clustering emerges naturally from the pretext tasks. Even without semantic labels, the network is forced to understand object geometry. Visualizations of the learned feature space show that semantically similar parts (e.g., the wings of an airplane or the legs of a table) cluster closely together in the embedding space, even when they are physically distant in the raw 3D input.
Furthermore, the method demonstrates high robustness to sparsity. It maintains competitive accuracy even when input point clouds are drastically downsampled to as few as 128 points. Ablation studies revealed that while both pretext tasks are valuable, the global distance prediction (regression) contributes slightly more to overall performance; however, the best results are achieved when both tasks are used simultaneously.
Conclusion and Future Work
This paper introduces a novel self-supervised pre-training method specifically tailored for data-scarce few-shot learning on point clouds. By leveraging cover-trees for hierarchical partitioning, the method generates robust embeddings that significantly boost performance across various downstream tasks.
While the method is sensitive to the hyperparameter $\epsilon$ (the expansion constant for the cover-tree), it represents a significant step forward in unsupervised 3D representation learning. Future work may explore additional pretext tasks that incorporate spectral information to further enrich the learned embeddings.
Summary updated