Class-Specific Channel Attention for Few-Shot Learning

謝君偉ICIP 2022

TL;DR

This paper tackles the challenge of "task distribution shift" in Few-Shot Learning (FSL) by introducing Class-Specific Channel Attention (CSCA). CSCA dynamically highlights discriminative feature channels for each class, leading to state-of-the-art performance on several FSL benchmarks.

What Problem Does This Paper Address?

Deep learning models excel when trained on large, well-annotated datasets. However, in real-world scenarios, acquiring such datasets can be impractical or impossible. Few-Shot Learning (FSL) aims to address this by enabling models to learn from very few examples of new, unseen categories. A major hurdle in FSL is the "task distribution shift" – the difference between the data distribution of the training (base) classes and the testing (novel) classes. This shift can cause models to focus on irrelevant or shared features, hindering their ability to generalize. Conventional methods often struggle to adapt pre-trained feature extractors to these novel distributions, leading to suboptimal performance.

Methods

The core contribution of this paper is the Class-Specific Channel Attention (CSCA) module. Unlike general attention mechanisms that learn global features, CSCA learns a unique weight vector for each class. This vector is trained to emphasize the most discriminative channels within the feature representations extracted by a pre-trained backbone network for that specific class.

Here's a breakdown of the approach:

  • Pre-trained Feature Backbone: The paper utilizes a pre-trained feature backbone (WRN-Net in their experiments) trained on a large base dataset. This backbone extracts feature vectors from input images.
  • CSCA Module: For each class, a trainable weight vector $W^n$ is introduced. This vector, with the same dimension as the number of channels in the feature vector, is learned to assign importance scores to each channel.
  • Feature Transformation: The extracted feature vector $E$ is then element-wise multiplied by the class-specific weight vector $W^n$ to produce a transformed feature vector $E'$. This process effectively "highlights" or "suppresses" certain channels based on their learned importance for that class.
  • Loss Function: The CSCA module is trained using a novel loss function that aims to maximize the intra-class distance (making features within the same class more spread out) and minimize the inter-class distance (making features from different classes more distinct). This is achieved by calculating prototypes (centroids) for each class and then optimizing based on the distances between these prototypes and the transformed feature vectors.
  • Inductive vs. Transductive Settings: The paper evaluates their method in both inductive (predictions made independently) and transductive (predictions consider all samples in a batch) FSL settings.

Datasets

The paper evaluates their method on four standard FSL benchmarks:

  • miniImagenet: 100 classes from ImageNet, split into 64 training, 16 validation, and 20 testing classes. Images are 84x84.
  • Tiered-ImageNet: A larger subset of ImageNet with images from 608 classes, split into 20 training, 6 validation, and 8 testing categories. Images are 84x84.
  • CIFAR-FS (CIFAR-100 Few-Shots): Based on CIFAR-100, split into 64 training, 16 validation, and 20 testing classes. Images are 32x32.
  • CUB-200-2011: A dataset of 200 bird species, split into 100 training, 50 validation, and 50 testing classes. Images are 84x84.

The authors emphasize that their experiments are conducted without using external datasets for training the feature backbone, aiming for a fairer comparison with other methods.

Key Insights and Ideas

  • Task Distribution Shift is Crucial: The paper strongly argues that the task distribution shift is a critical, often overlooked, problem in FSL, particularly concerning channel bias in feature representations.
  • Class-Specific Attention is Key: The core insight is that a "one-size-fits-all" attention mechanism is insufficient. Instead, learning class-specific channel importance allows the model to adapt more effectively to the nuances of each category.
  • Local vs. Global Features: CSCA focuses on learning local, class-specific features, contrasting with other attention methods that aim for global class features. This local focus is presented as more effective for FSL.
  • Efficient Computation: The CSCA module is designed to be computationally efficient, requiring few parameters and computations, making it practical for FSL tasks.
  • Cross-Domain Performance: The method shows stable and even superior performance in cross-domain settings, suggesting its robustness to significant shifts in data distribution.

Key Results

  • State-of-the-Art Performance: The proposed CSCA module achieves new state-of-the-art results on miniImagenet, Tiered-ImageNet, CIFAR-FS, and CUB-200-2011 in the inductive FSL setting.
  • Outperforms SoTA: Table 1 shows that CSCA outperforms all listed state-of-the-art algorithms across all tested benchmarks in the inductive setting.
  • Strong Cross-Domain Results: Table 2 demonstrates stable and competitive performance in cross-domain evaluations, with some values even exceeding in-domain results.
  • Improved Feature Separation: Visualizations (Figure 7) using t-SNE show that CSCA leads to better separation of feature vectors, with larger inter-class variance and smaller intra-class variance.
  • Efficient Training: The paper reports an average execution time of 0.44 seconds per run on an Nvidia GeForce RTX 2080 GPU, which includes training CSCA weights and classification.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while their method achieves excellent results, it's still important to fine-tune the feature backbone or extracted feature vectors with target data to fully mitigate the impact of task distribution shift.
  • The ablation study suggests that the raw feature vectors without any transformation are significantly impacted by the task distribution shift.
  • Future Work:
    • The authors plan to further evaluate CSCA on granularity shift tasks.
  • They intend to apply their method to other computer vision tasks, such as object detection in a few-shot setting.

Summary updated

BibTeX
@inproceedings{2022class,
  author = {謝君偉},
  title = {{Class-Specific Channel Attention for Few-Shot Learning}},
  booktitle = {ICIP},
  year = {2022},
}
Figure