What Makes for Effective Few-shot Point Cloud Classification?

Chuangguan Ye, Hongyuan Zhu, Yongbin Liao, Yanggang Zhang, Tao Chen, Jiayuan Fan

Abstract

Due to the emergence of powerful computing resources and large-scale annotated datasets, deep learning has seen wide applications in our daily life. However, most current methods require extensive data collection and retraining when dealing with novel classes never seen before. On the other hand, we humans can quickly recognize new classes by looking at a few samples, which motivates the recent popularity of few-shot learning (FSL) in machine learning communities. Most current FSL approaches work on 2D image domain, however, its implication in 3D perception is relatively under-explored. Not only needs to recognize the unseen examples as in 2D domain, 3D few-shot learning is more challenging with unordered structures, high intra-class variances, and subtle inter-class differences. Moreover, different architectures and learning algorithms make it difficult to study the effectiveness of existing 2D methods when migrating to the 3D domain. In this work, for the first time, we perform systematic and extensive studies of recent 2D FSL and 3D backbone networks for benchmarking few-shot point cloud classification, and we suggest a strong baseline and learning architectures for 3D FSL. Then, we propose a novel plug-and-play component called Cross-Instance Adaptation (CIA) module, to address the high intra-class variances and subtle inter-class differences issues, which can be easily inserted into current baselines with significant performance improvement. Extensive experiments on two newly introduced benchmark datasets, ModelNet40-FS and ShapeNet70-FS, demonstrate the superiority of our proposed network for 3D FSL.

Introduction

While deep learning has mastered 2D image recognition, 3D perception remains a "low-data" frontier. Most 3D models require massive annotated datasets, yet real-world applications like robotics or autonomous driving often require recognizing novel objects from just a few examples. This paper identifies three core challenges unique to 3D Few-Shot Learning (FSL): the unordered structure of point clouds, high intra-class variance (e.g., diverse chair designs), and subtle inter-class differences (e.g., similar geometric features between benches and chairs).

To address these, the authors first benchmarked existing 2D FSL algorithms (e.g., ProtoNet, MAML) against various 3D backbones (PointNet, RSCNN, DGCNN). They discovered that a graph-based backbone, DGCNN, paired with ProtoNet (Prototypical Networks), forms an exceptionally strong baseline for 3D tasks due to its ability to dynamically update connections in feature space.

Methodology: Cross-Instance Adaptation

To overcome the limitations of standard baselines, the researchers proposed the Cross-Instance Adaptation (CIA) module. This "plug-and-play" component is designed to be inserted into existing networks to refine feature representations on the fly.

The CIA module architecture integrates SCI and CIF components between the embedding network and the classifier.

The CIA module consists of two primary sub-modules:

  1. Self-Channel Interaction (SCI): This models correlations between different feature channels. Since different channels often capture distinct semantic parts (e.g., "legs" vs. "handles"), SCI helps the model focus on the most discriminative features to distinguish between similar classes.
  2. Cross-Instance Fusion (CIF): This addresses the "distribution shift" between labeled support samples and unlabeled query samples. It uses a meta-learner to re-weight features based on their similarity to other instances in the task, effectively aligning the feature space.

Experimental Results

To evaluate the method, the authors introduced two new benchmarks: ModelNet40-FS (30 training/10 testing classes) and ShapeNet70-FS (50 training/20 testing classes), ensuring no class overlap between training and testing sets.

t-SNE visualizations demonstrate how the CIA module (SCI and CIF) progressively tightens clusters and increases inter-class separation compared to the ProtoNet baseline.

The proposed method achieved significant performance gains:

  • ModelNet40-FS (5-way 1-shot): Reached 75.70% accuracy, a ~5.7% improvement over the standard ProtoNet baseline.
  • ShapeNet70-FS (5-way 1-shot): Reached 73.57% accuracy, outperforming other methods by a similar margin.

Visual analysis via t-SNE (Figure 3) confirms that the CIA module effectively rectifies the feature space, leading to tighter, more separable clusters. Despite the added complexity, the model maintains high inference speeds, making it viable for real-time applications.

Conclusion

This work provides the first systematic study of 3D few-shot point cloud classification. By identifying a powerful baseline (DGCNN + ProtoNet) and introducing the CIA module, the authors set new state-of-the-art benchmarks. Future work will explore more complex, cluttered scenes and the potential of self-supervised pre-training to further enhance feature extraction in low-data regimes.

Summary updated

Abstract

Due to the emergence of powerful computing resources and large-scale annotated datasets, deep learning has seen wide applications in our daily life. However, most current methods require extensive data collection and retraining when dealing with novel classes never seen before. On the other hand, we humans can quickly recognize new classes by looking at a few samples, which motivates the recent popularity of few-shot learning (FSL) in machine learning communities. Most current FSL approaches work on 2D image domain, however, its implication in 3D perception is relatively under-explored. Not only needs to recognize the unseen examples as in 2D domain, 3D few-shot learning is more challenging with unordered structures, high intra-class variances, and subtle inter-class differences. Moreover, different architectures and learning algorithms make it difficult to study the effectiveness of existing 2D methods when migrating to the 3D domain. In this work, for the first time, we perform systematic and extensive studies of recent 2D FSL and 3D backbone networks for benchmarking few-shot point cloud classification, and we suggest a strong baseline and learning architectures for 3D FSL. Then, we propose a novel plug-and-play component called Cross-Instance Adaptation (CIA) module, to address the high intra-class variances and subtle inter-class differences issues, which can be easily inserted into current baselines with significant performance improvement. Extensive experiments on two newly introduced benchmark datasets, ModelNet40-FS and ShapeNet70-FS, demonstrate the superiority of our proposed network for 3D FSL.
BibTeX
@inproceedings{ye2022what,
  author = {Chuangguan Ye and Hongyuan Zhu and Yongbin Liao and Yanggang Zhang and Tao Chen and Jiayuan Fan},
  title = {{What Makes for Effective Few-shot Point Cloud Classification?}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
  url = {https://openreview.net/forum?id=TL1Bb2TxzL},
}
Figure