Cross-Domain Few-Shot Incremental Learning for Point-Cloud Recognition

Yuwen Tan, Xiang XiangWACV 2024 · DOI

Introduction

Standard Few-Shot Class-Incremental Learning (FSCIL) aims to help models learn new categories from limited examples while retaining knowledge of base classes. However, real-world applications often face a significant domain shift: base classes are typically clean, synthetic 3D models, whereas incremental classes are noisy, occluded real-world scans. Existing methods often struggle with this shift, either forgetting base knowledge or failing to adapt to the new, noisy domain. This paper introduces a novel framework for Cross-Domain FSCIL in 3D point-cloud recognition, treating new objects as "out-of-distribution" (OOD) relative to a synthetic base model.

Methodology

The proposed approach utilizes a three-pronged strategy to address domain shift and catastrophic forgetting:

  1. OOD-Based Task Prediction: The system first acts as an OOD detector. By determining if a sample resembles the synthetic base data, it routes the input to either the base classifier or a domain-specific module for new classes.
  2. Scaled Cosine Classifier: Unlike standard linear classifiers with unbounded logits, the authors employ a scaled cosine classifier. Its bounded similarity scores provide a more reliable confidence signal for OOD detection.
  3. Two-Branch Architecture & Dual Replay: The model maintains a frozen base-class branch to prevent forgetting. A separate branch is fine-tuned for new classes. To preserve knowledge across multiple incremental sessions, a "Dual Replay" strategy is used, storing both physical examples (hard labels) and previous output scores (soft labels).
The proposed framework architecture featuring the OOD-based detection module and dual replay strategy.

Experimental Setup

The study evaluates synthetic-to-real transitions across several benchmarks:

  • M2S: ModelNet → ScanObjectNN
  • S2C: ShapeNet → CO3D
  • S2S: ShapeNet → ScanObjectNN

These benchmarks test the model's ability to generalize from clean synthetic data to noisy, real-world 3D scans.

Results and Discussion

The framework was evaluated using Harmonic Accuracy ($A_h$) to ensure a balanced assessment of base and new class performance. On the M2S benchmark, the method achieved a harmonic accuracy of 54.3%, significantly outperforming state-of-the-art FSCIL methods like FACT (11.9%) and ALICE (16.0%).

Accuracy metrics for different fine-tuning configurations, highlighting the balance between base and new class performance.

Key insights include:

  • Incremental Learning as an OOD Problem: Decoupling the identification of new classes from their classification prevents the confusion that typically leads to forgetting.
  • Logits Over Probabilities: Maximum logits from the cosine classifier were found to be superior to Softmax probabilities for OOD detection, as they better reflect model uncertainty.
  • Fine-tuning Depth: Freezing shallow layers while fine-tuning deeper, class-specific layers provides the optimal balance between stability and adaptability.

Conclusion

While the OOD detection strategy is highly effective, its reliance on a threshold ($\gamma$) may be challenging when domain shifts are minimal. Future work could explore generative replay to eliminate the need for a physical data buffer. Nevertheless, this framework sets a new standard for cross-domain incremental learning in 3D point-cloud recognition, achieving nearly 45% accuracy on new classes in final sessions—a 4-5x improvement over previous methods.

Summary updated

BibTeX
@inproceedings{xiang2024cross,
  author = {XIANG XIANG and Yuwen Tan and Xiang Xiang},
  title = {{Cross-Domain Few-Shot Incremental Learning for Point-Cloud Recognition}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
  doi = {10.1109/wacv57701.2024.00230},
}
Figure