Learning from Adversarial Features for Few-Shot Classification

Wei Shen, Ziqiang Shi, Jun Sun

Abstract

Many recent few-shot learning methods concentrate on designing novel model architectures. In this paper, we instead show that with a simple backbone convolutional network we can even surpass state-of-the-art classification accuracy. The essential part that contributes to this superior performance is an adversarial feature learning strategy that improves the generalization capability of our model. In this work, adversarial features are those features that can cause the classifier uncertain about its prediction. In order to generate adversarial features, we firstly locate adversarial regions based on the derivative of the entropy with respect to an averaging mask. Then we use the adversarial region attention to aggregate the feature maps to obtain the adversarial features. In this way, we can explore and exploit the entire spatial area of the feature maps to mine more diverse discriminative knowledge. We perform extensive model evaluations and analyses on miniImageNet and tieredImageNet datasets demonstrating the effectiveness of the proposed method.

Introduction

Few-shot classification aims to recognize novel categories given only a handful of labeled examples. The primary challenge in this domain is overfitting: standard deep learning models tend to be "lazy," focusing on the most prominent discriminative features (e.g., a dog's head) while ignoring the rest of the object. While sufficient for the training set, this narrow focus often fails to generalize to unseen classes where the most discriminative features might differ. This paper addresses the problem by forcing the model to explore the entire spatial area of an image, mining diverse knowledge through an adversarial feature learning strategy.

Proposed Method

The core of the approach is a training pipeline that enhances a simple CNN backbone without requiring complex meta-learning architectures. The process involves three main steps:

  1. Adversarial Region Location: During training, the model calculates the entropy of its prediction to measure uncertainty. It then computes the derivative of this entropy with respect to a spatial averaging mask ($M_0$). This identifies "adversarial regions"—areas that, if emphasized, would increase the model's uncertainty.
  2. Adversarial Attention: An attention mask ($M_a$) is created to highlight these difficult regions. Feature maps are aggregated using this mask to produce adversarial features ($x_a$).
  3. Multi-Scale Feature Learning: The network is optimized using a combined loss function. It learns from high-level structured information and low-level adversarial features simultaneously. This forces the model to "leave its comfort zone" and find diverse clues for classification across the entire image.
Overview of the training and test phases. During training, adversarial regions are identified via entropy loss to generate robust features.

Crucially, the adversarial mechanism is only active during training. For inference, the model uses standard global average pooling, ensuring no additional computational overhead when classifying new images.

Experimental Results

The method was evaluated on two standard benchmarks: miniImageNet (100 classes) and tieredImageNet (608 classes). The latter is particularly challenging as training and testing categories are more distinct, providing a rigorous test for true generalization.

The proposed method achieved state-of-the-art (SOTA) results at the time of publication:

  • miniImageNet: Achieved 63.28% (1-shot) and 78.70% (5-shot) accuracy, outperforming complex models like LEO and SNAIL.
  • tieredImageNet: Achieved 69.54% (1-shot) and 84.35% (5-shot) accuracy.

Beyond accuracy, the model demonstrated significantly higher resilience to adversarial noise compared to standard baselines, confirming that the features learned are inherently more robust.

Analysis and Insights

The success of this method stems from mining "blind spots." By treating adversarial perturbations as a diagnostic tool rather than just an attack, the authors identify regions the model would otherwise ignore.

Visualization of attention maps. The middle row shows standard attention, while the bottom row shows the proposed adversarial attention focusing on more diverse discriminative regions.

As shown in the visualization above, the adversarial attention (bottom row) encourages the model to look beyond the most obvious features, covering a wider spatial area of the target object. This results in a more robust and "smooth" metric space, which is vital for few-shot generalization. The study also highlights that a well-trained, simple backbone can often outperform more complex meta-learning algorithms.

Conclusion

This work demonstrates that adversarial feature learning is a powerful tool for improving generalization in few-shot classification. By forcing a simple CNN to learn from regions that cause uncertainty, the model develops a more comprehensive understanding of visual categories. Future work may involve automatically learning the scaling factors for entropy loss or integrating this robust feature extractor with task-specific adaptation methods.

Summary updated

Abstract

Many recent few-shot learning methods concentrate on designing novel model architectures. In this paper, we instead show that with a simple backbone convolutional network we can even surpass state-of-the-art classification accuracy. The essential part that contributes to this superior performance is an adversarial feature learning strategy that improves the generalization capability of our model. In this work, adversarial features are those features that can cause the classifier uncertain about its prediction. In order to generate adversarial features, we firstly locate adversarial regions based on the derivative of the entropy with respect to an averaging mask. Then we use the adversarial region attention to aggregate the feature maps to obtain the adversarial features. In this way, we can explore and exploit the entire spatial area of the feature maps to mine more diverse discriminative knowledge. We perform extensive model evaluations and analyses on miniImageNet and tieredImageNet datasets demonstrating the effectiveness of the proposed method.
BibTeX
@misc{shen2019learning,
  author = {Wei Shen and Ziqiang Shi and Jun Sun},
  title = {{Learning from Adversarial Features for Few-Shot Classification}},
  year = {2019},
  eprint = {1903.10225},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1903.10225},
}
Figure