Adversarial Feature Hallucination Networks for Few-Shot Learning

Kai Li, Yulun Zhang, Kunpeng Li, Yun Fu

Introduction

Few-Shot Learning (FSL) aims to bridge the gap between human-like learning from minimal examples and the data-intensive nature of modern deep neural networks. While traditional data augmentation techniques like image rotation or flipping provide some relief, they often fail to capture the underlying distribution of novel classes. Generative Adversarial Networks (GANs) offer a more powerful alternative by synthesizing new data. However, GANs in FSL face two primary hurdles: ensuring the synthetic data is discriminative enough for classification and maintaining sufficient diversity to avoid mode collapse.

Adversarial Feature Hallucination Networks

To address these challenges, the Adversarial Feature Hallucination Network (AFHN) operates directly in the high-level feature space rather than generating raw images. By utilizing a pre-trained backbone as a feature extractor, the model focuses on generating mathematical representations that are most relevant to the final classification task.

AFHN Architecture Overview

The core of AFHN is a conditional Wasserstein GAN (cWGAN). The generator $G$ is conditioned on a class "prototype"—the mean feature vector of the available support samples—to produce "hallucinated" features. The Wasserstein distance ensures stable training, while the conditioning ensures that the generated features remain relevant to the target class.

Regularization for Discriminability and Diversity

The primary innovation of AFHN lies in two novel regularizers designed to optimize the quality of hallucinated features:

  1. Classification Regularizer (CR): This loss function ensures that the generated features are discriminative. It forces the generator to produce features that exhibit high cosine similarity with their respective class prototypes and low similarity with others, directly aiding the few-shot classifier.
  2. Anti-collapse Regularizer (AR): To combat mode collapse, the AR penalizes the generator if it produces similar output features from distinct noise vectors. By maintaining a proportional relationship between input noise diversity and output feature diversity, the model is forced to explore a wider range of the class distribution.

Experimental Evaluation

AFHN was rigorously tested on standard benchmarks, including Mini-ImageNet, CUB-200, and CIFAR-100. On Mini-ImageNet, the model achieved a state-of-the-art 1-shot accuracy of 62.38%, outperforming previous GAN-based approaches like MetaGAN by significant margins.

Impact of Regularizers on Feature Distribution

Visual analysis using t-SNE confirms the efficacy of the proposed regularizers. As shown in the progression above, a standard cWGAN often suffers from mode collapse, where features for different classes overlap significantly. The addition of CR improves class separation, while the AR ensures that each class cluster is sufficiently spread out, representing a diverse set of "hallucinated" examples that better define class boundaries.

Conclusion and Future Work

AFHN demonstrates that hallucinating features is a highly effective strategy for few-shot learning, provided that both discriminability and diversity are explicitly managed. While the model shows exceptional performance on general datasets, its gains are more modest on fine-grained datasets like CUB-200, where low intra-class variance limits the "hallucination space." Future research will explore adapting these techniques for zero-shot learning and integrating more advanced backbone architectures.

Summary updated

BibTeX
@inproceedings{li2020adversarial,
  author = {Kai Li and Yulun Zhang and Kunpeng Li and Yun Fu},
  title = {{Adversarial Feature Hallucination Networks for Few-Shot Learning}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2020},
  doi = {10.1109/cvpr42600.2020.01348},
}
Figure