Abstract
Few-shot classification requires adapting knowledge learned from a large annotated base dataset to recognize novel unseen classes, each represented by few labeled examples. In such a scenario, pretraining a network with high capacity on the large dataset and then finetuning it on the few examples causes severe overfitting. At the same time, training a simple linear classifier on top of "frozen" features learned from the large labeled dataset fails to adapt the model to the properties of the novel classes, effectively inducing underfitting. In this paper we propose an alternative approach to both of these two popular strategies. First, our method pseudo-labels the entire large dataset using the linear classifier trained on the novel classes. This effectively "hallucinates" the novel classes in the large dataset, despite the novel categories not being present in the base database (novel and base classes are disjoint). Then, it finetunes the entire model with a distillation loss on the pseudo-labeled base examples, in addition to the standard cross-entropy loss on the novel dataset. This step effectively trains the network to recognize contextual and appearance cues that are useful for the novel-category recognition but using the entire large-scale base dataset and thus overcoming the inherent data-scarcity problem of few-shot learning. Despite the simplicity of the approach, we show that that our method outperforms the state-of-the-art on four well-established few-shot classification benchmarks.
Introduction
Few-shot classification (FSC) presents a fundamental challenge in computer vision: how to generalize from a data-rich "base" domain to a data-scarce "novel" domain. Traditional approaches typically fall into two categories: full model fine-tuning, which often leads to severe overfitting due to the high capacity of modern backbones relative to the few available examples; and transfer learning with frozen backbones, which avoids overfitting but results in underfitting as the model cannot adapt its internal representations to the specific nuances of novel classes.
This paper introduces Label Hallucination, a novel training paradigm that leverages the large-scale base dataset to facilitate full-model adaptation without the risk of overfitting. By "hallucinating" novel class labels within the base dataset, the model can learn task-specific features using a massive volume of auxiliary data.
Methodology
The Label Hallucination approach consists of a three-step pipeline designed to bridge the gap between base and novel domains.
1. Pre-training
A standard backbone network (e.g., ResNet-12) is first trained on the large base dataset using standard cross-entropy loss to learn general-purpose visual features.
2. Hallucination (Pseudo-labeling)
For a given few-shot task (an "episode"), a temporary linear classifier is trained on the few available novel examples (the support set). This classifier is then used to generate "soft" pseudo-labels for the entire base dataset. Even though the base images do not contain the novel classes, the classifier assigns probability distributions that capture similarities between base images and novel categories. For instance, a base image of a tiger might receive a high soft-label probability for a novel "domestic cat" class.
3. Full-Model Fine-tuning
The entire network—including the backbone and the classifier—is fine-tuned using a joint loss function. This function combines standard cross-entropy loss on the novel support set with a knowledge distillation loss applied to the pseudo-labeled base examples. This process allows the model to recognize contextual and appearance cues (e.g., shared backgrounds or textures) relevant to the novel classes by utilizing the scale of the base dataset as a non-parametric data augmentation tool.
Experimental Results
The proposed method was evaluated on four major few-shot benchmarks: miniImageNet, tieredImageNet, CIFAR-FS, and FC100.
As shown in the results, Label Hallucination consistently sets new state-of-the-art (SOTA) performance levels. On miniImageNet, the method achieves 68.28% accuracy in the 1-shot setting and 86.54% in the 5-shot setting. Similar gains are observed on tieredImageNet, where it reaches 73.34% (1-shot) and 87.68% (5-shot).
Ablation studies confirmed that using "soft" labels is critical; "hard" labels (assigning base images to a single novel class) significantly underperform. Furthermore, the results indicate that fine-tuning the entire network (backbone and classifier) provides a substantial boost—approximately 5% in 5-shot accuracy—compared to standard transfer learning with a frozen backbone.
Discussion and Conclusion
Label Hallucination effectively addresses the data scarcity problem in few-shot learning by repurposing disjoint base data. By forcing the model to find novel-class features within the base dataset, the method enables safe, full-model fine-tuning.
While the approach is computationally more intensive than frozen-backbone methods due to the per-episode fine-tuning, its performance gains are significant. Future work may explore the application of this hallucination strategy to broader transfer learning contexts beyond classification and investigate its robustness across larger domain gaps, such as transitioning from natural images to specialized medical or satellite imagery.
Summary updated