Memory-Augmented Relation Network for Few-Shot Learning

Jun He, Richang Hong, Xueliang Liu, Mingliang Xu, Zheng-Jun Zha, Meng Wang

Abstract

Few-shot learning (FSL) aims to recognize novel classes with minimal labeled data. Traditional metric-based methods often treat samples independently and rely on fixed distance metrics, ignoring the rich contextual relationships within a task. We present the Memory-Augmented Relation Network (MRN), which explicitly models these relationships using an episodic memory and a learnable relation module. By performing weighted information propagation, MRN enhances feature embeddings based on visual affinity, leading to more robust decision boundaries. Our approach achieves state-of-the-art performance on miniImageNet and tieredImageNet benchmarks.

Introduction

The core challenge of few-shot learning is the scarcity of labeled data. Most existing methods, such as Prototypical Networks, compare query images to support prototypes using fixed metrics like Euclidean distance. However, these methods suffer from two primary limitations: they ignore the "working context"—the relationships between support samples themselves—and they assume a rigid geometry in the feature space that may not align with the data distribution.

Methodology: Exploiting Working Context

The MRN architecture introduces a dynamic way to refine image representations by considering the entire task context.

Overview of the MRN approach compared to standard methods. (a) and (b) show typical query-support relations, while (c) illustrates the proposed embedding enhancement through neighborhood discovery and information propagation.

The system consists of four key components:

  • Feature Extractor: A standard convolutional backbone (Conv-4-64) that maps images into a latent feature space.
  • Episodic Memory: A short-term storage for feature embeddings, cleared and re-initialized for each new task.
  • Information Aggregation: For any image, the model identifies its $k$-nearest neighbors in memory and performs weighted information propagation. This updates the image's embedding by aggregating features from its neighbors based on learned visual affinity, effectively performing "manifold smoothing."
  • Relation Module ($g_\phi$): A learnable CNN that replaces fixed distance formulas, allowing the model to "learn to compare" samples in a way optimized for the specific feature space.

A significant advantage of MRN is its ability to operate transductively. By including unlabeled query images in the memory during propagation, the model can "borrow" information from the query set to further refine its internal representations.

Experimental Results

The authors evaluated MRN on the miniImageNet and tieredImageNet benchmarks, which are gold standards for few-shot learning research.

Key Performance Metrics

MRN demonstrated substantial improvements over previous state-of-the-art methods:

  • miniImageNet: Achieved 57.83% accuracy in 1-shot and 71.13% in 5-shot settings.
  • tieredImageNet: Reached 62.65% (1-shot) and 74.20% (5-shot) accuracy.

The gains were most pronounced in the 1-shot setting, where data is most limited and contextual information is most critical. Qualitative analysis via t-SNE visualizations confirmed that the information propagation step results in tighter, more distinct class clusters.

Discussion and Conclusion

The success of MRN highlights that "context is king" in few-shot scenarios. By looking at neighbors, the model can enhance weak feature embeddings with information from similar, stronger embeddings. However, there is a "Goldilocks" depth to this propagation; aggregating from too many "neighbors of neighbors" can cause embeddings to collapse and lose discriminative power.

While the current work uses a shallow backbone, future research will explore scaling MRN to deeper architectures like ResNet-101 and exploring "relation propagation," where relationship scores themselves are refined to further improve the manifold.

Summary updated

BibTeX
@inproceedings{he2020memory,
  author = {Jun He and Richang Hong and Xueliang Liu and Mingliang Xu and Zheng-Jun Zha and Meng Wang},
  title = {{Memory-Augmented Relation Network for Few-Shot Learning}},
  booktitle = {ACM Multimedia},
  year = {2020},
  doi = {10.1145/3394171.3413811},
}
Figure