Abstract
Few-shot learning features the capability of generalizing from a few examples. In this paper, we first identify that a discriminative feature space, namely a rectified metric space, that is learned to maintain the metric consistency from training to testing, is an essential component to the success of metric-based few-shot learning. Numerous analyses indicate that a simple modification of the objective can yield substantial performance gains. The resulting approach, called rectified metric propagation (ReMP), further optimizes an attentive prototype propagation network, and applies a repulsive force to make confident predictions. Extensive experiments demonstrate that the proposed ReMP is effective and efficient, and outperforms the state of the arts on various standard few-shot learning datasets.
Introduction
Few-shot learning (FSL) aims to teach models to recognize new objects from just a handful of examples, mimicking human-like adaptability. Most modern FSL methods utilize metric-based learning, where a model learns an embedding space where similar items are clustered together.
However, a significant "missing ingredient" in many top-tier models is metric inconsistency. Many existing approaches use a two-phase strategy: pre-training with a standard linear classifier (parametric) followed by testing with a nearest-neighbor approach (non-parametric). This disconnect creates an inconsistency—the feature space optimized for a linear layer is not necessarily optimal for nearest-neighbor matching. The authors found that this inconsistency alone can lead to a performance drop of up to 5% in recognition accuracy.
Methods: Rectified Metric Propagation (ReMP)
To address the metric inconsistency, the authors propose Rectified Metric Propagation (ReMP). The framework integrates two primary innovations designed to align training objectives with testing requirements.
1. Label-Aligned Cooperative Learning
Instead of choosing between global classification and local matching, ReMP performs both simultaneously. It employs a Global Matching loss to encourage fast convergence and transferability across classes, alongside a Local Matching loss to ensure the metric space is optimized for the episodic nearest-neighbor testing used in FSL. This cooperative objective ensures that the model's training and testing behaviors are perfectly aligned.
2. Contextualized Prototype Rectification
In standard FSL, a "prototype" (an average feature vector) is calculated for each class using only the few labeled support examples. ReMP adopts a transductive approach, utilizing unlabeled "query" images to iteratively refine these prototypes.
To make this refinement robust, ReMP introduces Repulsive Attention. If a query image is significantly dissimilar from a prototype, the model applies a "repulsive force" to push it away, preventing it from "polluting" the prototype's average. This mechanism prevents decision boundary drift and leads to more confident predictions. By stacking multiple layers of self-attention, the model iteratively "rectifies" its understanding of each class within the context of the specific task.
Experiments and Results
ReMP was evaluated on three standard few-shot learning benchmarks: miniImageNet, tieredImageNet, and CIFAR-FS. The results demonstrate that ReMP consistently outperforms previous state-of-the-art (SoTA) methods:
- miniImageNet: Achieved 79.25% accuracy in 1-shot tasks and 87.01% in 5-shot tasks using a ResNet backbone.
- tieredImageNet: Reached 82.01% (1-shot) and 87.92% (5-shot).
- Efficiency: Unlike meta-learning methods that require complex fine-tuning for new tasks, ReMP operates in a direct, feed-forward manner, offering high efficiency.
Conclusion and Future Work
ReMP demonstrates that simply fixing the training-testing metric gap can yield substantial performance gains. By combining global and local matching with an attentive, repulsive prototype rectification mechanism, the model creates a highly discriminative feature space.
While effective, ReMP's transductive nature means it currently requires access to the entire query set at once, which may limit its use in streaming scenarios. Future work may explore different repulsive strategies or more complex attention architectures to further sharpen decision boundaries in the rectified metric space.
Summary updated