MemREIN: Rein the Domain Shift for Cross-Domain Few-Shot Learning

Yi Xu, Lichen Wang, Yizhou Wang, Can Qin, Yulun Zhang, Yun Fu

Abstract

Cross-Domain Few-Shot Learning (CD-FSL) faces significant challenges due to the domain gap between source and target datasets. Standard normalization techniques often discard fine-grained discriminative details while removing domain-specific styles. We present MemREIN, a framework that utilizes Instance Normalization (IN) alongside a novel "memorized restitution" module. This module captures and re-integrates essential discriminative information stored in a global memory bank. Evaluated across diverse target domains, MemREIN consistently improves classification accuracy, demonstrating its effectiveness as a backbone-agnostic enhancement for CD-FSL.

Introduction

Few-shot learning (FSL) aims to recognize new categories with minimal examples. However, CD-FSL introduces a "domain shift" where the model must generalize from a source domain (e.g., ImageNet) to a distinct target domain (e.g., satellite imagery or specific fine-grained categories). The primary hurdle is that domain-specific "styles" (lighting, texture, color) often mask the underlying "content" features. While Instance Normalization (IN) can mitigate style interference, it frequently deletes the subtle details necessary for distinguishing similar classes. MemREIN addresses this by treating the information discarded during normalization as a valuable resource to be salvaged and re-integrated.

Methods

The MemREIN (Memorized, REstitution, and Instance Normalization) framework is designed to preserve discriminative details while benefiting from the domain-robustness of normalization.

The MemREIN architecture, featuring Instance Normalization, a Memory Bank for feature restitution, and Reverse Contrastive Loss.

As illustrated in the architecture above, the framework consists of three main components:

  1. Instance Normalization (IN): Applied at the channel level, IN removes sample-specific noise and style variations, a technique borrowed from style transfer to improve domain generalization.
  2. Memorized Restitution: Normalization is inherently lossy. MemREIN captures the "residual" data thrown away by IN and uses a channel-attention mechanism to separate it into "discriminative" and "general" information. The discriminative components are stored in a Memory Bank across training iterations and then "restituted" (added back) to the refined features.
  3. Reverse Contrastive Loss ($L_{rcl}$): To ensure the restitution module learns effectively, this loss function forces the restituted features ($F^+$) to be highly separable and discriminative, while ensuring that features containing discarded general info ($F^-$) remain less capable of class differentiation.

Experiments and Results

The researchers evaluated MemREIN using a "leave-one-out" cross-domain setting, pre-training on mini-ImageNet and testing on four target domains: CUB (birds), Cars, Places, and Plantae. The framework was tested as a plug-and-play enhancement for three popular backbones: MatchingNet (MNet), RelationNet (RNet), and Graph Neural Networks (GNN).

t-SNE visualizations showing feature clustering for GNN, GNN+IN, and GNN+MemREIN across different tasks.

Key Findings

  • Significant Accuracy Gains: MemREIN consistently outperformed state-of-the-art baselines. Notably, on the 5-way 1-shot task for the Cars dataset, MemREIN improved accuracy by 16.43% over the standard baseline.
  • Visual Clustering: As shown in the t-SNE visualizations, while standard IN helps, the addition of MemREIN creates much tighter and more distinct clusters for different classes in the target domain, providing clear visual proof of its discriminative power.
  • Stability via Memory: By using a memory bank to store long-term feature maps, the model maintains a stable, global view of discriminative features, which is crucial for preventing overfitting in few-shot scenarios.

Conclusion and Future Work

MemREIN demonstrates that the "trash" discarded by standard normalization is actually a "treasure" of fine-grained details. By re-integrating this information through a memorized restitution mechanism, the framework effectively reins in the domain shift. While highly effective, future research could address hyperparameter sensitivity regarding the loss balancing parameter ($\lambda$) and explore the scalability of this approach with larger architectures like Vision Transformers (ViTs).

Summary updated

Abstract

Few-shot learning aims to enable models generalize to new categories (query instances) with only limited labeled samples (support instances) from each category. Metric-based mechanism is a promising direction which compares feature embeddings via different metrics. However, it always fail to generalize to unseen domains due to the considerable domain gap challenge. In this paper, we propose a novel framework, MemREIN, which considers Memorized, Restitution, and Instance Normalization for cross-domain few-shot learning. Specifically, an instance normalization algorithm is explored to alleviate feature dissimilarity, which provides the initial model generalization ability. However, naively normalizing the feature would lose fine-grained discriminative knowledge between different classes. To this end, a memorized module is further proposed to separate the most refined knowledge and remember it. Then, a restitution module is utilized to restitute the discrimination ability from the learned knowledge. A novel reverse contrastive learning strategy is proposed to stabilize the distillation process. Extensive experiments on five popular benchmark datasets demonstrate that MemREIN well addresses the domain shift challenge, and significantly improves the performance up to 16.43% compared with state-of-the-art baselines.
BibTeX
@inproceedings{xu2022memrein,
  author = {Yi Xu and Lichen Wang and Yizhou Wang and Can Qin and Yulun Zhang and Yun Fu},
  title = {{MemREIN: Rein the Domain Shift for Cross-Domain Few-Shot Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/505},
}
Figure