RotMix and RotCom: Two Regularization Strategies with Contrastive Learning for Few-shot Classification

Kunkun Ma, Baopeng Zhang, Teng Zhu, Junnan Zou, Jianping Fan

Abstract

Few-shot learning (FSL) aims to recognize new classes by adapting prior knowledge with extremely limited examples. The key challenge lies in eliminating the induction bias between SEEN and UNSEEN classes. We propose a novel embedding model called MixNet, which encodes class diversity to eliminate induction bias and evolves prior knowledge through contrastive learning. Furthermore, we introduce two data augmentation methods, RotMix and RotCom, involving random rotation and random grayscale of image patches. These encourage the model to focus on multiple discriminative regions rather than just the most salient ones. Extensive experiments on miniImageNet and tieredImageNet verify that MixNet with RotMix and RotCom significantly boosts performance compared to existing metric-based methods.

Introduction

Few-shot learning (FSL) represents a significant challenge in computer vision: training a model to recognize novel categories from only one or five examples. A primary hurdle is induction bias, where a model trained on "seen" base classes relies on global patterns or salient features that do not generalize to "unseen" classes. While research often focuses on loss functions or meta-learning architectures, this paper highlights the importance of efficient data augmentation to force the learning of robust, transferable representations.

MixNet Architecture

The proposed MixNet addresses induction bias through a specialized embedding model. It utilizes a ResNet-12 backbone followed by a "Global Adaptor" to capture diverse class patterns.

The MixNet architecture utilizes a Global Adaptor and contrastive learning to bridge the gap between support and query sets.

MixNet is first pre-trained on base classes using standard cross-entropy loss and then fine-tuned using Contrastive Self-Supervised Learning (CSSL). By employing an InfoNCE loss, the model learns to associate different augmented versions of the same image while distinguishing between different images. This process helps the model adapt to the distribution of "unseen" classes without requiring large-scale labeled data.

Rotational Augmentation: RotMix and RotCom

To prevent the model from relying solely on the most salient image regions, the authors introduce two novel augmentation strategies: RotMix and RotCom.

  • RotMix: Building on the concept of CutMix, RotMix pastes a patch from one image onto another. Uniquely, it applies a random rotation (0°, 90°, 180°, or 270°) and random grayscaling to the patch.
  • RotCom: This variation combines crops specifically from within the same class, serving as a regularization technique.
Visual comparison of various data augmentation techniques, including the proposed RotMix and RotCom methods.

By distorting the most obvious features through rotation or color removal, these methods force the model to identify more subtle, secondary discriminative regions, leading to better generalization.

Experimental Results

The researchers evaluated MixNet on several benchmarks, including miniImageNet, tieredImageNet, and fine-grained datasets like Stanford Dogs.

  • miniImageNet: MixNet with RotMix achieved 70.66% accuracy in 1-shot tasks, a significant improvement over the Rethink-Distill baseline.
  • tieredImageNet: The model reached 73.12% (1-shot) and 86.86% (5-shot) accuracy, establishing a new state-of-the-art.
  • Fine-Grained Classification: On the Stanford Dogs dataset, RotMix improved performance over ProtoNet by nearly 3%, demonstrating its ability to capture local details necessary for distinguishing similar categories.

Ablation studies confirmed that both contrastive learning and the RotMix/RotCom augmentations are essential components; removing either led to a notable decrease in classification accuracy.

Conclusion

MixNet, combined with RotMix and RotCom, provides a powerful framework for few-shot classification. By integrating contrastive learning with advanced rotational and grayscale augmentations, the model effectively mitigates induction bias and avoids over-fitting on limited data. Future work may explore adaptive augmentation strategies to further refine which image regions are most beneficial to transform for specific tasks.

Summary updated

Abstract

<title>Abstract</title> Few-shot learning (FSL) aims to recognize new classes by adapting prior knowledge with extremely limited few-shot examples. The key challenge lies in eliminating the induction bias between SEEN classes and UNSEEN classes. Performance improvements mainly focus on enhancing feature representations or category embeddings, or designing new loss functions, while largely ignoring efficient data augmentation techniques for enhanced training. In light of this, we propose a novel embedding model called MixNet, which encodes the diversity of classes to eliminate induction bias, and evolves prior knowledge through contrastive learning. It captures sketchy patterns and focuses on detailed characteristics. Our contrastive adaptation model implicitly learns the UNSEEN embeddings to alleviate the over-fitting problem with limited labeled data.\textbf{ } Furthermore, we propose two data augmentation methods named RotMix and RotCom to exploit the extra information, which can boost the generalization ability of few-shot models. The proposed RotMix and RotCom involve random rotation and random grayscale of any patch in the original image, which encourage the model to pay attention to multiple discriminative regions of the object rather than the most salient discriminative region. Extensive experiments are conducted on widely used few-shot recognition benchmarks, including miniImageNet and tieredImageNet, verifying that our MixNet with RotMix and RotCom can boost significant performance compared with both classic and recent popular metric-based methods. Besides, we also compare the proposed methods against other augmentation methods such as Cutout, Mixup, and CutMix. Experimental results reflect the competing performance of RotMix and RotCom.
BibTeX
@inproceedings{ma2023rotmix,
  author = {Kunkun Ma and Baopeng Zhang and Teng Zhu and Junnan Zou and Jianping Fan},
  title = {{RotMix and RotCom: Two Regularization Strategies with Contrastive Learning for Few-shot Classification}},
  booktitle = {Research Square},
  year = {2023},
  doi = {10.21203/rs.3.rs-3779888/v1},
}
Figure