Revisiting Few-Shot Learning from a Causal Perspective

Guoliang Lin, Yongheng Xu, Hanjiang Lai, Jian Yin

Abstract

Few-shot learning with $N$-way $K$-shot scheme is an open challenge in machine learning. Many metric-based approaches have been proposed to tackle this problem, e.g., the Matching Networks and CLIP-Adapter. Despite that these approaches have shown significant progress, the mechanism of why these methods succeed has not been well explored. In this paper, we try to interpret these metric-based few-shot learning methods via causal mechanism. We show that the existing approaches can be viewed as specific forms of front-door adjustment, which can alleviate the effect of spurious correlations and thus learn the causality. This causal interpretation could provide us a new perspective to better understand these existing metric-based methods. Further, based on this causal interpretation, we simply introduce two causal methods for metric-based few-shot learning, which considers not only the relationship between examples but also the diversity of representations. Experimental results demonstrate the superiority of our proposed methods in few-shot classification on various benchmark datasets. Code is available in https://github.com/lingl1024/causalFewShot.
BibTeX
@article{lin2024revisiting,
  author = {Guoliang Lin and Yongheng Xu and Hanjiang Lai and Jian Yin},
  title = {{Revisiting Few-Shot Learning from a Causal Perspective}},
  journal = {IEEE Transactions on Knowledge and Data Engineering},
  year = {2024},
  url = {https://openreview.net/forum?id=P7gx0yUgXH},
}

Loading related content...

Figure