What Makes for Effective Few-shot Point Cloud Classification?

Chuangguan Ye, Hongyuan Zhu, Yongbin Liao, Yanggang Zhang, Tao Chen, Jiayuan Fan

Abstract

Due to the emergence of powerful computing resources and large-scale annotated datasets, deep learning has seen wide applications in our daily life. However, most current methods require extensive data collection and retraining when dealing with novel classes never seen before. On the other hand, we humans can quickly recognize new classes by looking at a few samples, which motivates the recent popularity of few-shot learning (FSL) in machine learning communities. Most current FSL approaches work on 2D image domain, however, its implication in 3D perception is relatively under-explored. Not only needs to recognize the unseen examples as in 2D domain, 3D few-shot learning is more challenging with unordered structures, high intra-class variances and subtle inter-class differences. Moreover, different architectures and learning algorithms make it difficult to study the effectiveness of existing 2D methods when migrating to the 3D domain. In this work, for the first time, we perform systematic and extensive studies of recent 2D FSL and 3D backbone networks for benchmarking few-shot point cloud classification, and we suggest a strong baseline and learning architectures for 3D FSL. Then, we propose a novel plug-and-play component called Cross-Instance Adaptation (CIA) module, to address the high intra-class variances and subtle inter-class differences issues, which can be easily inserted into current baselines with significant performance improvement. Extensive experiments on two newly introduced benchmark datasets, ModelNet40-FS and ShapeNet70-FS, demonstrate the superiority of our proposed network for 3D FSL.

Introduction: The Challenge of 3D Few-Shot Learning

While deep learning has mastered 2D image recognition, 3D perception remains a "data-hungry" frontier. In real-world scenarios, we often encounter new objects with only a few examples available for training. However, 3D point clouds present unique hurdles compared to 2D images: they are unordered, irregular, and exhibit significant intra-class variance. For instance, a mailbox can look vastly different from another mailbox depending on its design, yet look subtly similar to a trash can (inter-class similarity).

Visualization of the "Mailbox Problem": (a) t-SNE comparison showing tighter clusters in 2D ImageNet vs. scattered 3D ShapeNet features. (b) Normalized Euclidean distance to class prototypes.

Existing 2D few-shot algorithms do not always migrate well to 3D. While 2D models benefit from massive pre-training on ImageNet, 3D models are often trained on smaller datasets, leading to poor embedding spaces where features from the same category are widely scattered. This "Mailbox Problem" highlights that the primary challenge in 3D FSL is learning a robust enough space to handle extreme shape variety.

Methodology

The researchers conducted a two-stage study: a comprehensive benchmark of existing methods and the introduction of a novel architectural component.

1. Benchmarking 3D FSL

The study evaluated several 2D FSL algorithms (e.g., ProtoNet, MAML, and RelationNet) across various 3D backbones (PointNet, PointNet++, DGCNN). A key finding was that DGCNN (Dynamic Graph CNN) is the superior backbone for 3D FSL because it dynamically updates a graph of local neighbors, capturing geometric structures better than pointwise methods.

2. The Cross-Instance Adaptation (CIA) Module

To address the scattered feature problem, the authors proposed the Cross-Instance Adaptation (CIA) module, a plug-and-play component designed to align support and query features.

The proposed 3D few-shot learning framework incorporating the Cross-Instance Adaptation (CIA) module.

The CIA module consists of two main parts:

  • Self-Channel Interaction (SCI): This models correlations between different feature channels. Since different channels capture distinct semantic traits (e.g., "has a handle" vs. "is flat"), SCI helps the model focus on the most discriminative features for a specific task.
  • Cross-Instance Fusion (CIF): Instead of processing support and query samples independently, CIF uses a meta-learner to re-weight features based on their relationships. It aligns query features with relevant support prototypes to reduce distribution shift.

Experiments and Results

To properly evaluate 3D FSL, the authors introduced two new benchmark splits: ModelNet40-FS (30 training/10 testing classes) and ShapeNet70-FS (50 training/20 testing classes).

The proposed "ProtoNet + DGCNN + CIA" framework set a new state-of-the-art:

  • Accuracy Boost: On ModelNet40-FS (5-way 1-shot), the CIA module improved the baseline from 69.95% to 75.70%.
  • Plug-and-Play Versatility: The CIA module provided a consistent 2-5% accuracy increase regardless of the underlying FSL algorithm (MAML, RelationNet, etc.).
  • Efficiency: Despite the added complexity, the model maintains high inference speeds, exceeding 80 tasks per second on a standard GPU.

Conclusion

This work provides the first systematic study of FSL for 3D point clouds, identifying metric-learning with graph-based backbones as the strongest baseline. While the CIA module effectively mitigates distribution shift, 3D FSL remains limited by the scale of annotated data. Future directions include exploring self-supervised pre-training on massive unlabeled datasets to build more robust initial embedding spaces and testing methods on noisy real-world scans.

Summary updated

Abstract

Due to the emergence of powerful computing resources and large-scale annotated datasets, deep learning has seen wide applications in our daily life. However, most current methods require extensive data collection and retraining when dealing with novel classes never seen before. On the other hand, we humans can quickly recognize new classes by looking at a few samples, which motivates the recent popularity of few-shot learning (FSL) in machine learning communities. Most current FSL approaches work on 2D image domain, however, its implication in 3D perception is relatively under-explored. Not only needs to recognize the unseen examples as in 2D domain, 3D few-shot learning is more challenging with unordered structures, high intra-class variances and subtle inter-class differences. Moreover, different architectures and learning algorithms make it difficult to study the effectiveness of existing 2D methods when migrating to the 3D domain.In this work, for the first time, we perform systematic and extensive studies of recent 2D FSL and 3D backbone networks for benchmarking few-shot point cloud classification, and we suggest a strong baseline and learning architectures for 3D FSL. Then, we propose a novel plug-and-play component called Cross-Instance Adaptation (CIA) module, to address the high intra-class variances and subtle inter-class differences issues, which can be easily inserted into current baselines with significant performance improvement. Extensive experiments on two newly introduced benchmark datasets, ModelNet40-FS and ShapeNet70-FS, demonstrate the superiority of our proposed network for 3D FSL.
BibTeX
@inproceedings{ye2022what,
  author = {Chuangguan Ye and Hongyuan Zhu and Yongbin Liao and Yanggang Zhang and Tao Chen and Jiayuan Fan},
  title = {{What Makes for Effective Few-shot Point Cloud Classification?}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
  url = {https://openreview.net/forum?id=TL1Bb2TxzL},
}
Figure