Conditional Self-Supervised Learning for Few-Shot Classification

Yuexuan An, Hui Xue, Xingyu Zhao, Lu Zhang

Abstract

Few-shot classification (FSC) aims to recognize novel categories from extremely limited data. While self-supervised learning (SSL) is often used to enhance feature representations, it frequently suffers from representation bias in data-scarce regimes. This paper introduces Conditional Self-Supervised Learning (CSS), a three-stage framework that leverages supervised labels to guide self-supervised feature discovery. By making SSL conditional on supervised priors, CSS prevents the model from learning irrelevant "shortcuts" and achieves state-of-the-art performance on standard FSC benchmarks without requiring additional unlabeled data.

Introduction

The core challenge of few-shot classification is learning robust, generalizable features from only a few examples. To bridge the gap between human-like learning and deep learning models, researchers have integrated self-supervised learning (SSL) as an auxiliary task. However, standard SSL techniques are typically data-hungry. In the low-data regime of FSC, SSL can "go rogue," focusing on low-level patterns like color histograms or edge continuity rather than semantic information. This representation bias can degrade performance on the primary classification task.

To address this, the authors propose Conditional Self-Supervised Learning (CSS). The key idea is to use supervised information as a "prior" to constrain the self-supervised branch, ensuring it explores feature variations that are useful for the downstream task.

Methodology

The CSS framework consists of a three-stage training pipeline designed to harmonize supervised and self-supervised objectives.

The three-stage CSS framework: (a) Supervised Pre-training, (b) Conditional Self-supervised Training, and (c) Meta-training with Fusion Distillation.

1. Supervised Pre-training

In the first stage, a standard feature extractor $f_\theta$ is trained using all available labeled data from the base classes. This establishes a baseline supervised representation.

2. Conditional Self-Supervised Training

This stage introduces a second feature extractor $g_\xi$, trained via a self-supervised method (e.g., SimSiam). Unlike vanilla SSL, a Conditional Module forces $g_\xi$ to align with the supervised features from $f_\theta$. By minimizing the distance between these representations ($L_{cond}$), the supervised model acts as a teacher, guiding the self-supervised branch to ignore irrelevant shortcuts and focus on meaningful semantic structures.

3. Meta-training with Fusion Distillation

Finally, the knowledge from both the supervised ($f_\theta$) and self-supervised ($g_\xi$) branches is fused into a final embedding network $h_\phi$. A Fusion Distillation (FD) method, utilizing a Graph Convolutional Network (GCN), aggregates information across samples to enrich the final feature representation used for meta-learning.

Experimental Results

The CSS model was evaluated on three standard benchmarks: CIFAR-FS, CUB-200 (fine-grained birds), and mini-ImageNet.

Comparison of accuracy across different datasets and training configurations. CSS consistently outperforms vanilla SSL and standard supervised baselines.

Key Performance Metrics

CSS consistently achieved state-of-the-art results in both 1-shot and 5-shot scenarios:

  • mini-ImageNet: Reached 68.08% accuracy in 5-shot tasks, significantly higher than traditional Prototypical Networks (64.69%).
  • CUB-200: Achieved 81.84% in 5-shot tasks, outperforming other SSL-based FSC methods by 5-10%.
  • CIFAR-FS: Reached 74.59% in 5-shot tasks.

Insights

Ablation studies confirmed that "vanilla" self-supervision often performs worse than simple supervised learning in few-shot settings. The Conditional Module was identified as the primary driver of performance gains, proving that guided exploration is essential when data is limited. The best results were obtained by integrating both paradigms: the supervised branch captures inter-class differences, while the self-supervised branch mines intra-sample structures.

Conclusion and Future Work

CSS demonstrates that self-supervision, when properly conditioned on supervised priors, can significantly boost few-shot classification performance. By preventing representation bias, the framework ensures that learned features are both semantically rich and task-relevant.

Future research directions include addressing the computational overhead of the three-stage process and the GCN fusion. Additionally, exploring automated hyperparameter tuning for the loss balancing and scaling the framework to larger backbones like Vision Transformers (ViTs) could further enhance its utility in diverse computer vision applications.

Summary updated

Abstract

How to learn a transferable feature representation from limited examples is a key challenge for few-shot classification. Self-supervision as an auxiliary task to the main supervised few-shot task is considered to be a conceivable way to solve the problem since self-supervision can provide additional structural information easily ignored by the main task. However, learning a good representation by traditional self-supervised methods is usually dependent on large training samples. In few-shot scenarios, due to the lack of sufficient samples, these self-supervised methods might learn a biased representation, which more likely leads to the wrong guidance for the main tasks and finally causes the performance degradation. In this paper, we propose conditional self-supervised learning (CSS) to use auxiliary information to guide the representation learning of self-supervised tasks. Specifically, CSS leverages supervised information as prior knowledge to shape and improve the learning feature manifold of self-supervision without auxiliary unlabeled data, so as to reduce representation bias and mine more effective semantic information. Moreover, CSS exploits more meaningful information through supervised and the improved self-supervised learning respectively and integrates the information into a unified distribution, which can further enrich and broaden the original representation. Extensive experiments demonstrate that our proposed method without any fine-tuning can achieve a significant accuracy improvement on the few-shot classification scenarios compared to the state-of-the-art few-shot learning methods.
BibTeX
@inproceedings{an2021conditional,
  author = {Yuexuan An and Hui Xue and Xingyu Zhao and Lu Zhang},
  title = {{Conditional Self-Supervised Learning for Few-Shot Classification}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.24963/ijcai.2021/295},
}
Figure