ReMP: Rectified Metric Propagation for Few-Shot Learning

Yang Zhao, Chunyuan Li, Ping Yu, Changyou Chen

Abstract

Few-shot learning features the capability of generalizing from a few examples. In this paper, we first identify that a discriminative feature space, namely a rectified metric space, that is learned to maintain the metric consistency from training to testing, is an essential component to the success of metric-based few-shot learning. Numerous analyses indicate that a simple modification of the objective can yield substantial performance gains. The resulting approach, called rectified metric propagation (ReMP), further optimizes an attentive prototype propagation network, and applies a repulsive force to make confident predictions. Extensive experiments demonstrate that the proposed ReMP is effective and efficient, and outperforms the state of the arts on various standard few-shot learning datasets.

Introduction

Few-shot learning (FSL) aims to recognize novel classes using only a handful of labeled examples. Most successful FSL methods rely on metric learning, which seeks to learn an embedding space where similar objects are clustered together. However, a significant challenge remains: Metric Inconsistency.

Many modern frameworks employ a two-phase approach where a standard parametric linear classifier is used during training, but a non-parametric nearest-neighbor approach using class "prototypes" is used during testing. This switch creates a discrepancy—the feature space optimized for a linear classifier is not necessarily optimal for nearest-neighbor matching. This inconsistency can lead to a performance drop of up to 5% in absolute recognition accuracy.

Methodology

To address metric inconsistency, we propose Rectified Metric Propagation (ReMP), which is built upon three core pillars.

1. Label-Aligned Cooperative Learning

ReMP ensures metric consistency by optimizing both a Global Loss and a Local Loss simultaneously. The global loss matches features to a set of learnable global weights, while the local loss matches query images to task-specific prototypes. This dual objective forces the model to maintain a consistent metric logic from training through to inference.

Overview of the ReMP framework showing the alignment between global and local metric spaces.

2. Contextualized Label Embeddings

In standard FSL, prototypes are typically the mean of available labeled examples. ReMP "rectifies" these prototypes by incorporating information from unlabeled query images. Using a self-attention mechanism, prototypes evolve based on the distribution of the specific task data they are classifying, a process known as transductive refinement.

3. Repulsive Attention

Standard attention mechanisms can be susceptible to noise from dissimilar query images. ReMP introduces a "repulsive force" to mitigate this. If a query image is significantly different from a prototype, the repulsive mechanism pushes it away, ensuring that prototype refinement is driven by relevant data and that decision boundaries remain sharp.

The process of prototype rectification using repulsive forces to sharpen decision boundaries.

Experiments and Results

ReMP was evaluated on three standard benchmarks: miniImageNet, tieredImageNet, and CIFAR-FS. The model consistently outperformed previous state-of-the-art (SoTA) methods:

  • miniImageNet: Achieved 79.25% (1-shot) and 87.01% (5-shot) accuracy using a ResNet backbone.
  • tieredImageNet: Reached 82.01% (1-shot) accuracy, a significant improvement over existing models.

Ablation studies confirmed that the "Global Matching" component is essential. Removing it led to a substantial drop in performance, proving that a parametric global view is vital for learning a discriminative feature space, even if it is not used directly at test time.

Conclusion

ReMP demonstrates that the consistency of the metric space is a critical factor in FSL performance. By aligning global and local learning objectives and refining prototypes with a novel repulsive attention mechanism, ReMP achieves superior results across major benchmarks. While its transductive nature implies a dependency on seeing the full query set, its efficiency and effectiveness mark a significant step forward in closing the gap in few-shot recognition.

Summary updated

Abstract

Few-shot learning features the capability of generalizing from a few examples. In this paper, we first identify that a discriminative feature space, namely a rectified metric space, that is learned to maintain the metric consistency from training to testing, is an essential component to the success of metric-based few-shot learning. Numerous analyses indicate that a simple modification of the objective can yield substantial performance gains. The resulting approach, called rectified metric propagation (ReMP), further optimizes an attentive prototype propagation network, and applies a repulsive force to make confident predictions. Extensive experiments demonstrate that the proposed ReMP is effective and efficient, and outperforms the state of the arts on various standard few-shot learning datasets.
BibTeX
@inproceedings{zhao2021remp,
  author = {Yang Zhao and Chunyuan Li and Ping Yu and Changyou Chen},
  title = {{ReMP: Rectified Metric Propagation for Few-Shot Learning}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  url = {https://openreview.net/forum?id=QDP58REAhkj},
}
Figure