Variational Few-Shot Learning

Jian Zhang, Chenglong Zhao, Bingbing Ni, Minghao Xu, Xiaokang Yang

Abstract

Few-shot learning (FSL) aims to recognize novel classes with minimal labeled examples. Current metric-learning approaches predominantly rely on point estimates, such as class prototypes, which are highly susceptible to noise and outliers when data is scarce. This paper introduces a variational Bayesian framework that represents classes as full distributions rather than single vectors. By explicitly modeling uncertainty through mean and variance estimation, our approach achieves greater robustness and sets new performance standards on benchmarks like miniImageNet and Omniglot.

Introduction

Deep learning's success typically hinges on large-scale labeled datasets. However, in many real-world scenarios, only a few examples per category are available. Few-shot learning (FSL) addresses this by learning to generalize from limited data. The dominant paradigm in FSL is metric learning, where models learn an embedding space to cluster similar objects.

Most existing methods use point estimates—for instance, Prototypical Networks represent a class by the mean vector of its support samples. As shown in Figure 1, this approach has significant drawbacks. First, it is vulnerable to noise; a single outlier in a 1-shot or 5-shot setting can drastically shift the prototype. Second, point estimates lack interpretability regarding the "spread" or variance of a class in high-dimensional space.

Comparison between point estimates and the proposed variational distribution method.

Methods

We propose a Variational Few-Shot Learning framework that maps images to multivariate Gaussian distributions instead of single points.

1. Distribution Generation and Framework

The architecture consists of a feature extractor ($F$) and a raw generator ($G$). For each input image, the model predicts a mean ($\mu$) and a variance ($\sigma^2$), representing the central location and the associated uncertainty of the sample in the embedding space.

Overview of the Variational Few-Shot Learning framework.

2. Variance-Weighted Aggregation

In multi-shot scenarios (e.g., 5-shot), the model aggregates individual sample distributions into a class distribution. Rather than simple averaging, it employs a variance-weighted linear combination. Samples with lower predicted variance—indicating higher confidence or "cleaner" features—are given more weight, while noisy or boundary samples are suppressed.

3. Objective Function

Training is guided by the Evidence Lower Bound (ELBO), comprising two main terms:

  • Recognition Loss ($L_R$): Maximizes the likelihood that target samples belong to their respective estimated class distributions.
  • Intra-class KL Divergence ($L_{intra}$): Acts as a regularizer to ensure that distributions of samples within the same class are consistent and compact.

Experiments and Results

The framework was evaluated on three major benchmarks: Omniglot, miniImageNet, and Cluttered Omniglot.

Key Results

  • miniImageNet: Using a ResNet backbone, the method achieved 61.23% accuracy on the 5-way 1-shot task, a substantial improvement over the ~49% achieved by standard Prototypical Networks.
  • Omniglot: The model reached near-perfect accuracy of 99.67% for 5-way 1-shot tasks.
  • Robustness: Stability analysis revealed that the variational approach is significantly less sensitive to the specific choice of support samples compared to point-based methods.
  • Segmentation: On Cluttered Omniglot, the probabilistic approach improved the Intersection over Union (IOU) across various levels of noise and distractors.

Discussion and Conclusion

By treating uncertainty as a feature, our variational framework allows the network to distinguish between reliable and noisy dimensions in the embedding space. This probabilistic distance metric generalizes Euclidean distance by weighting features by their inverse variance.

While the Gaussian assumption and diagonal covariance are computationally efficient, the framework provides a robust foundation for reliable few-shot classification and segmentation. Future work may explore multimodal distributions to capture more complex class structures as the number of available samples increases.

Summary updated

Abstract

We propose a variational Bayesian framework for enhancing few-shot learning performance. This idea is motivated by the fact that single point based metric learning approaches are inherently noise-vulnerable and easy-to-be-biased. In a nutshell, stochastic variational inference is invoked to approximate bias-eliminated class specific sample distributions. In the meantime, a classifier-free prediction is attained by leveraging the distribution statistics on novel samples. Extensive experimental results on several benchmarks well demonstrate the effectiveness of our distribution-driven few-shot learning framework over previous point estimates based methods, in terms of superior classification accuracy and robustness.
BibTeX
@inproceedings{zhang2019variational,
  author = {Jian Zhang and Chenglong Zhao and Bingbing Ni and Minghao Xu and Xiaokang Yang},
  title = {{Variational Few-Shot Learning}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00177},
}
Figure