This paper, recently appearing on arXiv, tackles a major bottleneck in 3D computer vision: the "class-specific bias" that plagues few-shot learning. By moving away from traditional supervised pretraining and embracing self-supervision, the authors present a more robust way to segment 3D point clouds with minimal data.
TL;DR
The authors propose a few-shot point cloud segmentation framework that replaces biased, fully-supervised pretraining with contrastive self-supervision using a learnable augmentor. Combined with a Multi-Resolution Attention (MRA) module that efficiently captures local and global features, the approach achieves state-of-the-art results and shows significant promise for niche, real-world industrial applications like CAD/CAM segmentation.
What Problem Does This Paper Address?
Semantic segmentation of 3D point clouds usually requires massive, hand-labeled datasets. Few-shot learning (FSL) aims to solve this by learning to identify new classes from just a handful of examples.
However, most FSL methods rely on a "pretrain-then-transfer" paradigm. They pretrain a feature extractor on a large labeled dataset (like S3DIS) using supervised learning. The authors argue this creates a feature extraction bias: the model learns features optimized for the pretrain classes, which don't necessarily generalize well to the novel classes encountered during few-shot tasks. Furthermore, labeling 3D data is notoriously difficult, making large-scale supervised pretraining impractical for many specialized industrial fields.
Methods
The authors introduce three key innovations to solve these issues:
- Contrastive Self-Supervision with a Learnable Augmentor: Instead of using labels during pretraining, the model learns by comparing different "views" of the same point cloud. They integrate a trainable augmentor (inspired by PointAugment) that automatically optimizes point-level and shape-level deformations. This forces the encoder to learn universal, point-wise features that are class-agnostic and thus more generalizable.
- Multi-Resolution Attention (MRA): Standard attention mechanisms in 3D are computationally expensive and often get "distracted" by redundant background points. The MRA module selectively attends to the $K$-nearest neighbors (for local geometric details) and the farthest points (for global shape context). This dual-focus approach captures essential features while ignoring distractive noise.
- Intra-Class Centering Multi-Prototype: In few-shot learning, classes are often represented by "prototypes" (mean feature vectors). To handle the inherent sparsity and complexity of point clouds, the authors use a centering loss to ensure that multiple prototypes for the same class are tightly clustered in the feature space, improving discrimination between similar-looking objects.
Datasets
The model was evaluated on three primary datasets:
- S3DIS: A standard large-scale indoor scene dataset used for benchmarking.
- ShapeNet: A dataset of 3D CAD models used primarily to train the learnable augmentor.
- phcf (Practical CAD/CAM): A specialized dataset containing 3D machining features like "pockets," "holes," "chamfers," and "fillets." This represents a "real-world" small-data scenario where traditional supervised pretraining often fails due to overfitting.
Key Insights and Ideas
- Self-Supervision > Supervision for FSL: The most striking insight is that class-agnostic features (learned without labels) are actually better for few-shot generalization than features learned via supervised classification. This is because the self-supervised model doesn't "overfit" its perception to a specific set of categories.
- Selective Attention is Efficient: You don't need to look at every point to understand a scene. By focusing only on the "nearest" and "farthest" points, the MRA module achieves better performance with lower computational overhead than vanilla window-based attention.
- Domain-Specific Pretraining: The authors found that self-supervised pretraining on the same domain (even without labels) is incredibly powerful. For the CAD/CAM dataset, using self-supervision on the target domain improved results by over 17% compared to supervised pretraining on a different domain.
Key Results
- S3DIS Performance: The proposed method achieved 54.47% mIoU on 2-way 1-shot tasks, outperforming the previous state-of-the-art (attMPTI) which sat at 53.07%.
- CAD/CAM Success: In the practical case study (phcf dataset), the model significantly outperformed baselines, proving its effectiveness in industrial settings where data is scarce and shapes are highly similar.
- Ablation Studies: Removing the contrastive pretraining or the MRA module led to a consistent drop in accuracy, confirming that both the training strategy and the architecture are vital to the performance gains.
Limitations and Future Work
- Training Speed: The authors acknowledge that their framework is slightly slower to train than some baselines due to the added complexity of feature concatenation in the MRA module.
- Complexity: The pipeline involves several stages (learnable augmentation, contrastive pretraining, MRA, and graph-based label propagation), which might make it harder to implement or tune compared to simpler end-to-end models.
- Future Directions: The authors suggest that this self-supervised approach could be extended to other data-efficient paradigms, such as incremental learning, where the model must learn new classes over time without forgetting old ones.
Reference: Wang, J., Zhu, H., et al. "Few-Shot Point Cloud Semantic Segmentation via Contrastive Self-Supervision and Multi-Resolution Attention." arXiv:2302.10501v1 [cs.CV].
Summary updated