Improving Adversarially Robust Few-shot Image Classification with Generalizable Representations

Junhao Dong, Yuan Wang, Jianhuang Lai, Xiaohua Xie

Introduction

Few-Shot Image Classification (FSIC) aims to recognize novel object categories from a limited number of labeled examples. While deep learning has made significant strides in this area, models remain highly vulnerable to adversarial attacks—small, often imperceptible perturbations that lead to incorrect predictions. In few-shot scenarios, this vulnerability is exacerbated because the sparse data makes it difficult to define robust decision boundaries.

Adversarial attacks on few-shot classifiers and the conceptual difference between standard and robust decision boundaries.

Existing approaches typically rely on meta-learning, which involves training on thousands of sampled "tasks." However, these methods often overfit to specific training distributions, leading to poor performance when encountering adversarial attacks on unseen classes or in cross-domain settings (e.g., general objects to medical imagery). This paper proposes a non-meta-learning baseline that focuses on learning generalizable, robust representations.

Methodology

The authors move away from task-based meta-learning, opting instead for a direct embedding-learning approach enhanced by three key components:

1. Adversarial-Aware Mechanism ($L_{aa}$)

To improve representation stability, the framework incorporates a self-supervised auxiliary task. The model is trained to distinguish between "legitimate" and "adversarial" feature vectors. This process encourages the feature extractor to internalize the characteristics of adversarial transformations, resulting in more resilient embeddings.

2. Adversarial-Reweighted Training ($L_{ar}$)

Standard adversarial training often treats all perturbations equally. The authors introduce a composite weighting factor that prioritizes "stronger" or more disruptive attacks. This factor is calculated based on the number of iterations required to flip the model's prediction and the actual variation in loss. By focusing on high-impact examples, the model develops a more robust boundary against sophisticated attacks.

3. Feature Purifier ($L_{fp}$)

During the inference phase, a "plug-and-play" feature purifier module is employed. This post-processing step maps noisy adversarial features back toward the distribution of clean, legitimate features, further stabilizing classification on novel tasks.

Experimental Results

The proposed method was evaluated on standard benchmarks, including miniImageNet and CIFAR-FS, as well as cross-domain transfer tasks.

Performance Gains

The approach achieved a new state-of-the-art (SOTA) for robust FSIC. On the 1-shot miniImageNet benchmark, it outperformed previous meta-learning methods (like R-MAML) by 8.2% on clean images and 7.7% under PGD adversarial attacks. The model maintained significantly higher confidence and accuracy even as the attack strength increased.

Generalization and Transferability

A key strength of this method is its ability to generalize across domains. In cross-domain evaluations (e.g., training on miniImageNet and testing on CIFAR-FS), the adversarial-aware module significantly improved accuracy compared to meta-learning frameworks, which often struggle when the label space changes entirely.

Conclusion and Future Work

By prioritizing high-quality feature representations over complex meta-learning schedules, this work demonstrates that a well-designed baseline can achieve superior robustness and generalization. While the current feature purifier is architecture-dependent and the iterative generation of adversarial examples adds computational overhead, the results provide a strong foundation for future research into black-box attack resilience and more flexible purification modules.

Summary updated

Abstract

Few-Shot Image Classification (FSIC) aims to recognize novel image classes with limited data, which is significant in practice. In this paper, we consider the FSIC problem in the case of adversarial examples. This is an extremely challenging issue because current deep learning methods are still vulnerable when handling adversarial examples, even with massive labeled training samples. For this problem, existing works focus on training a network in the meta-learning fashion that depends on numerous sampled few-shot tasks. In comparison, we propose a simple but effective baseline through directly learning generalizable representations without tedious task sampling, which is robust to unforeseen adversarial FSIC tasks. Specifically, we introduce an adversarial-aware mechanism to establish auxiliary supervision via feature-level differences between legitimate and adversarial examples. Furthermore, we design a novel adversarial-reweighted training manner to alleviate the imbalance among adversarial examples. The feature purifier is also employed as post-processing for adversarial features. Moreover, our method can obtain generalizable representations to remain superior transferability, even facing cross-domain adversarial examples. Extensive experiments show that our method can significantly outperform state-of-the-art adversarially robust FSIC methods on two standard benchmarks.
BibTeX
@inproceedings{dong2022improving,
  author = {Junhao Dong and Yuan Wang and Jianhuang Lai and Xiaohua Xie},
  title = {{Improving Adversarially Robust Few-shot Image Classification with Generalizable Representations}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.00882},
}
Figure