Abstract
This paper presents an effective few-shot point cloud semantic segmentation approach for real-world applications. Existing few-shot segmentation methods on point cloud heavily rely on the fully-supervised pretrain with large annotated datasets, which causes the learned feature extraction bias to those pretrained classes. However, as the purpose of few-shot learning is to handle unknown/unseen classes, such class-specific feature extraction in pretrain is not ideal to generalize into new classes for few-shot learning. Moreover, point cloud datasets hardly have a large number of classes due to the annotation difficulty. To address these issues, we propose a contrastive self-supervision framework for few-shot learning pretrain, which aims to eliminate the feature extraction bias through class-agnostic contrastive supervision. Specifically, we implement a novel contrastive learning approach with a learnable augmentor for a 3D point cloud to achieve point-wise differentiation, so that to enhance the pretrain with managed overfitting through the self-supervision. Furthermore, we develop a multi-resolution attention module using both the nearest and farthest points to extract the local and global point information more effectively, and a center-concentrated multi-prototype is adopted to mitigate the intra-class sparsity. Comprehensive experiments are conducted to evaluate the proposed approach, which shows our approach achieves state-of-the-art performance. Moreover, a case study on practical CAM/CAD segmentation is presented to demonstrate the effectiveness of our approach for real-world applications.
Introduction
Semantic segmentation of 3D point clouds is a critical task in computer vision, yet it typically requires massive, hand-labeled datasets. Few-shot learning (FSL) offers a solution by enabling models to identify new classes from only a handful of examples. However, current FSL methods often suffer from "feature extraction bias." By pretraining on large labeled datasets using supervised learning, models become optimized for specific pretrain classes, failing to generalize effectively to novel classes. Additionally, the high cost of labeling 3D data makes large-scale supervised pretraining impractical for specialized industrial applications.
Methodology
The proposed framework addresses these challenges through three primary innovations: contrastive self-supervision, multi-resolution attention, and intra-class centering.
Contrastive Self-Supervision with a Learnable Augmentor
To eliminate class-specific bias, the model utilizes a self-supervised pretraining phase. Instead of relying on labels, the model learns by comparing different augmented "views" of the same point cloud. A trainable augmentor, inspired by PointAugment, automatically optimizes point-level and shape-level deformations. This forces the encoder to learn universal, point-wise features that are class-agnostic and highly generalizable.
Multi-Resolution Attention (MRA)
Standard attention mechanisms in 3D are often computationally expensive and prone to distraction by redundant background points. The MRA module selectively attends to the $K$-nearest neighbors to capture local geometric details and the farthest points to capture global shape context. This dual-focus approach ensures the model captures essential features while ignoring distractive noise.
Intra-Class Centering Multi-Prototype
To handle the inherent sparsity of point clouds, the authors employ an intra-class centering strategy. By using a centering loss, multiple prototypes representing the same class are tightly clustered in the feature space, improving the model's ability to discriminate between similar-looking objects.
Experiments and Results
Datasets and Evaluation
The model was evaluated across three diverse datasets:
- S3DIS: A standard large-scale indoor scene benchmark.
- ShapeNet: Used primarily for training the learnable augmentor.
- phcf (Practical CAD/CAM): A specialized dataset containing 3D machining features (e.g., pockets, holes, chamfers), representing a real-world small-data scenario.
Performance Metrics
On the S3DIS dataset, the proposed method achieved 54.47% mIoU on 2-way 1-shot tasks, surpassing the previous state-of-the-art (attMPTI at 53.07%). In the CAD/CAM case study, the model significantly outperformed baselines, demonstrating its utility in industrial settings where data is scarce and geometric features are highly similar. Qualitative results show that the proposed method produces segmentation masks that are much closer to the ground truth than standard baselines.
Key Insights and Conclusion
A major finding of this research is that class-agnostic features learned via self-supervision are superior for few-shot generalization compared to features learned through traditional supervised classification. Furthermore, domain-specific self-supervised pretraining proved highly effective; for the CAD/CAM dataset, pretraining on the target domain improved results by over 17% compared to cross-domain supervised pretraining.
While the framework involves increased complexity and slightly slower training times due to the MRA module's feature concatenation, the gains in accuracy and generalizability are substantial. Future work may extend this self-supervised approach to incremental learning scenarios, where models must adapt to new classes over time without catastrophic forgetting.
Summary updated