Towards A Conceptually Simple Defensive Approach for Few-shot classifiers Against Adversarial Support Samples

Yi Xiang Marcus Tan, Penny Chong, Jiamei Sun, Ngai-Man Cheung, Yuval Elovici, Alexander BinderarXiv 2021 · OpenReview

Introduction

Few-Shot Learning (FSL) has emerged as a powerful paradigm for recognizing novel objects with minimal labeled data. However, this flexibility introduces a significant security vulnerability. Recent research by teams from SUTD, Ben-Gurion University, and the University of Oslo highlights a critical "integrity attack" where an adversary poisons the support samples used at inference time. This paper proposes a novel, attack-agnostic defense mechanism based on the principle of self-similarity to detect such adversarial samples.

Problem Formulation: Integrity Attacks in FSL

In traditional deep learning, adversarial attacks typically target test-time query images. In FSL, the model's behavior is dynamically conditioned on a "support set." An adversary can craft perturbations for these support samples that, while imperceptible to humans, fundamentally alter the classifier's internal representation of a class. This leads to consistent misclassification of legitimate query images, effectively "blinding" the model. Given that FSL often operates in open-world environments where users provide their own data, this threat is highly practical.

Proposed Methodology: Split-Filter-Compare

The authors introduce a "split-filter-compare" strategy designed to exploit the fragility of adversarial perturbations compared to the inherent self-similarity of clean data.

Overview of the proposed split-filter-compare defense mechanism.

The defense process involves three primary steps:

  1. Auxiliary Splitting: The support set for a given class is randomly partitioned into an auxiliary support set ($S_{aux}^c$) and an auxiliary query set ($Q_{aux}^c$).
  2. Filtering via FPA: The auxiliary support set is processed through a Feature-space Preserving Autoencoder (FPA). Unlike standard autoencoders, the FPA is specifically trained to maintain the internal feature representations of the few-shot classifier while stripping away high-frequency adversarial noise.
  3. Adversarial Scoring ($U_{adv}$): The system calculates an adversarial score by comparing the model's logits for the auxiliary query set when using the original vs. the filtered auxiliary support. A significant shift in logits indicates the presence of adversarial samples, as clean samples exhibit high self-similarity and remain stable under filtering.

Experimental Evaluation

The defense was evaluated on two benchmark datasets: miniImageNet (MI) and CUB (Caltech-UCSD Birds). The researchers tested the approach against three prominent FSL architectures: RelationNet (RN), Prototypical Networks (PN), and Cross-Attention Networks (CAN). Attacks were generated using Projected Gradient Descent (PGD) and CW-SGD.

Results and Discussion

The proposed FPA-based detection achieved exceptional performance, often reaching AUROC scores exceeding 0.99. The method significantly outperformed standard Out-of-Distribution (OOD) baselines like ODIN and Isolation Forests, which typically require more data than is available in a few-shot context.

Detection performance across various attack settings (vanilla and lower-strength variants) for different FSL architectures.

Key findings include:

  • Robustness to Attack Strength: The defense remains highly effective across varying perturbation magnitudes ($\epsilon$), as shown in the comparative results for vanilla and lower-strength attack variants.
  • Logit-Based Detection: Utilizing raw model outputs (logits) proved more sensitive for detection than relying on hard label changes.
  • Transferability: The study confirmed that PGD-based support set attacks are highly transferable, making them a potent threat that necessitates robust defenses like the one proposed.

Conclusion

This work demonstrates that the very flexibility of Few-Shot Learning can be turned into a defensive advantage. By leveraging self-similarity and feature-preserving filtering, the proposed "split-filter-compare" approach provides a conceptually simple yet highly effective shield against adversarial support samples. Future research will focus on optimizing computational overhead for edge deployment and improving sensitivity to single-sample poisoning within larger support sets.

Summary updated

Abstract

Few-shot classifiers have been shown to exhibit promising results in use cases where user-provided labels are scarce. These models are able to learn to predict novel classes simply by training on a non-overlapping set of classes. This can be largely attributed to the differences in their mechanisms as compared to conventional deep networks. However, this also offers new opportunities for novel attackers to induce integrity attacks against such models, which are not present in other machine learning setups. In this work, we aim to close this gap by studying a conceptually simple approach to defend few-shot classifiers against adversarial attacks. More specifically, we propose a simple attack-agnostic detection method, using the concept of self-similarity and filtering, to flag out adversarial support sets which destroy the understanding of a victim classifier for a certain class. Our extended evaluation on the miniImagenet (MI) and CUB datasets exhibit good attack detection performance, across three different few-shot classifiers and across different attack strengths, beating baselines. Our observed results allow our approach to establishing itself as a strong detection method for support set poisoning attacks. We also show that our approach constitutes a generalizable concept, as it can be paired with other filtering functions. Finally, we provide an analysis of our results when we vary two components found in our detection approach.
BibTeX
@misc{tan2021towards,
  author = {Yi Xiang Marcus Tan and Penny Chong and Jiamei Sun and Ngai-Man Cheung and Yuval Elovici and Alexander Binder},
  title = {{Towards A Conceptually Simple Defensive Approach for Few-shot classifiers Against Adversarial Support Samples}},
  year = {2021},
  eprint = {2110.12357},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.12357},
}
Figure