Abstract
Few-shot learning (FSL) aims to adapt deep learning models to new categories using only a handful of labeled examples, a task often hampered by the risk of overfitting. This paper introduces VI-Net, a generative framework that addresses data scarcity by "hallucinating" synthetic feature vectors. By employing Variational Inference to map limited support samples into a structured latent distribution, VI-Net generates diverse features that sharpen classifier decision boundaries. Experimental results on mini-ImageNet and CUB benchmarks demonstrate that VI-Net achieves state-of-the-art performance among augmentation-based methods.
Introduction
Deep learning models typically require vast amounts of labeled data to generalize effectively. In Few-Shot Learning (FSL), models must adapt using only a small "support set" (e.g., 1 or 5 samples). The primary obstacle is the high risk of overfitting, as sparse data makes it difficult for classifiers to establish robust decision boundaries for unseen "query" samples.
While data augmentation is a standard remedy, traditional methods often rely on transferring intra-class variations (such as pose or lighting) from base categories. This approach can be imprecise; slight misalignments in the transfer direction can lead to generated samples that drift from the true class center, potentially degrading model performance.
Methodology: The VI-Net Framework
The proposed VI-Net architecture consists of four primary modules: a feature extractor ($F$), a distribution encoder ($E$), a feature decoder ($D$), and a cosine-similarity classifier ($C$).
Training and Inference Workflow
The framework operates in two distinct stages:
- Pre-training: A feature extractor and cosine classifier are trained on a large "base" dataset to establish general visual representations.
- Base-training (Generative Module): With the feature extractor frozen, the encoder $E$ learns to map features into a low-dimensional latent space. Rather than simple point mapping, $E$ estimates the parameters ($\mu, \sigma^2$) of a Gaussian distribution for each category.
Variational Inference and Hallucination
To ensure the latent space is representative, the model employs a KL-divergence loss ($L_{lat}$). This aligns the "posterior" distribution, derived from the support samples, with a "prior" distribution learned from the broader training set.
Once the distribution is established, the model samples random vectors from this space and passes them through the decoder $D$ to "hallucinate" new feature vectors ($\tilde{f}$). During inference, these synthetic features augment the original support samples to fine-tune the cosine classifier, effectively filling gaps in the feature space.
Experimental Evaluation
VI-Net was evaluated on two benchmark datasets: mini-ImageNet (general object recognition) and CUB (fine-grained bird species classification).
Key Results
- mini-ImageNet: VI-Net achieved 61.05% accuracy in 1-shot tasks and 78.60% in 5-shot tasks, outperforming baseline classifiers and competing augmentation methods like Delta-Encoder.
- CUB: Performance was notably strong, with 74.76% (1-shot) and 86.84% (5-shot) accuracy. The success on fine-grained data suggests that Variational Inference is particularly effective when intra-class differences are subtle.
- Ablation Studies: The necessity of the three loss functions—classification ($L_{cla}$), reconstruction ($L_{rec}$), and latent constraint ($L_{lat}$)—was confirmed. Removing $L_{lat}$ led to a significant accuracy drop, as generated features lacked the diversity needed to generalize.
Discussion and Key Insights
VI-Net's effectiveness stems from its focus on feature-level hallucination rather than image generation. This approach is computationally efficient and directly targets the classifier's requirements. By treating categories as distributions, the model generates "diverse but plausible" variations, which sharpens the decision boundary and provides the classifier with a more comprehensive view of the class manifold.
Limitations and Future Work
While VI-Net performs well, 1-shot scenarios remain challenging, as estimating a reliable distribution from a single data point is inherently difficult. Future research may explore incorporating more sophisticated prior knowledge from base classes to assist in 1-shot distribution estimation, potentially by learning a generalized "variance map" applicable across similar categories.
Summary updated