Abstract
Few-shot learning (FSL) aims to recognize novel categories from extremely limited data. However, deep learning models often suffer from "representation deficiency" in these scenarios, where high-dimensional parameter spaces lead to overfitting or poor generalization. This paper introduces MCRNet (Complemented Representations Network), a meta-learning framework that utilizes variational inference to embed images into a probabilistic latent space. By sampling from this distribution, the model complements missing information, allowing expressive architectures like ResNet-12 to achieve state-of-the-art performance on standard benchmarks.
Introduction
Deep learning typically requires massive datasets to learn robust features. In contrast, humans can learn new concepts from a single example. Few-shot learning (FSL) attempts to bridge this gap. A significant challenge in FSL is representation deficiency: with only one or five images, calculating meaningful gradients in a high-dimensional space is mathematically difficult.
Existing methods often resort to simpler, less expressive features to avoid overfitting. However, this creates a bottleneck where the model lacks the power to handle complex tasks. MCRNet addresses this by treating the latent space as a distribution rather than a fixed vector, effectively "filling in the blanks" of missing data through probabilistic modeling.
Methodology: The MCRNet Framework
MCRNet bridges the gap between low-data inputs and high-dimensional feature spaces through a three-stage architecture:
- Encoder: Raw images are processed through convolutional layers to extract initial low-dimensional features.
- Variational Latent Space: Instead of a deterministic feature vector, MCRNet maps inputs to a distribution defined by mean $\mu$ and variance $\sigma$. By sampling latent codes ($z$), the model handles task ambiguity and complements the inherent information deficiency of small datasets.
- Decoder and Base-Learner: These latent codes are decoded into a high-dimensional parameter space. For final classification, the framework employs robust "base-learners" with convex objectives, such as Support Vector Machines (SVM) and Ridge Regression (RR).
The system is trained using an episodic meta-learning strategy, alternating between an "inner loop" (optimizing the base-learner for specific tasks) and an "outer loop" (optimizing the meta-learner for generalizability across tasks).
Experimental Evaluation
The authors evaluated MCRNet on three major benchmarks: miniImageNet, CIFAR-FS, and FC100.
Quantitative Results
MCRNet achieved state-of-the-art (SOTA) performance, particularly in 5-shot scenarios:
- miniImageNet: Reached 80.34% accuracy, significantly outperforming previous models like LEO (77.59%) and MTL (75.5%).
- CIFAR-FS: Achieved 86.8% accuracy, demonstrating the effectiveness of the ResNet-12 backbone when paired with complemented representations.
Qualitative Insights
Visual analysis of the reconstructed samples shows that MCRNet produces "sharper" and more detailed representation outputs compared to baseline models. This suggests that the probabilistic latent space successfully captures relevant textures and structural details necessary for accurate classification despite the limited input data.
Discussion and Conclusion
MCRNet challenges the traditional view that high-dimensional latent spaces necessarily lead to overfitting in few-shot scenarios. By using variational inference to provide "extra information," the model stabilizes training and improves accuracy. While the choice of latent dimensions (found to be optimal at 64) and computational overhead remain areas for further optimization, MCRNet provides a powerful template for applying deep architectures to data-scarce domains. Future work may extend this "complemented representation" philosophy to natural language processing and other complex modalities.
Summary updated