Learning Task-aware Local Representations for Few-shot Learning

Chuanqi Dong, Wenbin Li, Jing Huo, Zheng Gu, Yang Gao

Abstract

Few-shot learning (FSL) aims to recognize new categories with minimal labeled examples. While traditional methods rely on global image-level embeddings, these often fail to capture the subtle, task-specific nuances required for fine-grained recognition. This paper introduces ATL-Net (Adaptive Task-aware Local Representations Network), which shifts the focus to local patches. By employing a novel episodic attention mechanism with adaptive thresholding, ATL-Net dynamically selects discriminative local features based on the specific task context. Our approach achieves state-of-the-art results on standard benchmarks and fine-grained datasets like Stanford Dogs and CUB-200, while maintaining a lightweight parameter footprint.

Introduction: The Need for Task-Awareness

The core challenge in few-shot learning is generalizing from a "support set" of just one or five images to a "query" image. Most existing approaches condense an entire image into a single global vector. However, a representation optimized for general classification may overlook the specific details needed to distinguish between similar classes in a new task.

For instance, distinguishing a "sparrow" from a "hawk" might require focusing on size and beak shape, whereas distinguishing two types of sparrows might require focusing on plumage patterns. The authors argue that the importance of local image patches is task-dependent. Existing local representation (LR) models often treat all patches equally or use rigid comparison metrics like $k$-nearest neighbors, failing to adapt to the specific set of classes at hand.

Methodology: ATL-Net

ATL-Net addresses these issues through two main components: an embedding module and an adaptive episodic attention module.

Overview of the ATL-Net framework showing the flow from support/query images to final classification scores via the Adaptive Episodic Attention Module.

1. Embedding Module ($F_\Theta$)

A convolutional neural network extracts local feature maps from images. Each map represents a set of local patches that capture spatial information.

2. Adaptive Episodic Attention Module ($F_A$)

This is the central innovation of ATL-Net. Unlike previous "rank-based" methods that select a fixed number of similar patches (e.g., top-k), ATL-Net uses value-based selection:

  • Adaptive Thresholding: A small Multi-Layer Perceptron (MLP) predicts a unique similarity threshold for each query patch. This allows the model to dynamically determine how many support patches are relevant for a given query.
  • Sparse Attention: To filter out background noise (like grass or sky), a sparse attention mechanism "mutes" non-discriminative patches and amplifies those that define the category within the current task's context.

Experimental Results

The model was evaluated on four benchmarks: miniImageNet for general FSL, and Stanford Dogs, Stanford Cars, and CUB-200 for fine-grained FSL.

Performance on Fine-Grained Datasets

ATL-Net demonstrates its greatest strength in fine-grained scenarios where local details are paramount. As shown in the comparison below, it consistently outperforms existing state-of-the-art (SOTA) methods.

Comparison of ATL-Net against SOTA methods on fine-grained datasets. ATL-Net shows significant improvements, particularly in the Stanford Dogs 5-shot setting.

On Stanford Dogs, ATL-Net outperformed the next best method by 5.39% in the 1-shot setting and 9.69% in the 5-shot setting. On miniImageNet, it achieved 54.30% (1-shot) and 73.22% (5-shot) accuracy, surpassing previous benchmarks.

Key Insights

  • Contextual Recognition: ATL-Net mimics human behavior by looking for features that distinguish an object from current alternatives, rather than looking at it in isolation.
  • Efficiency: Despite its high performance, ATL-Net is remarkably lightweight, using only 0.117M parameters—significantly fewer than models like GNN (1.6M) or GCR (1.7M).
  • Ablation Studies: Removing the adaptive threshold module led to a performance drop of over 1%, validating that learning patch-specific thresholds is critical for success.

Conclusion and Future Work

ATL-Net proves that task-aware local representations are highly effective for few-shot learning, especially in fine-grained domains. While the current model uses a simple MLP for thresholding and a standard 4-layer convolutional backbone, future work could explore deeper backbones like ResNet or explicitly modeling spatial dependencies between patches. Overall, ATL-Net provides a robust and efficient framework for adaptive, local-feature-based recognition in low-data regimes.

Summary updated

Abstract

Few-shot learning for visual recognition aims to adapt to novel unseen classes with only a few images. Recent work, especially the work based on low-level information, has achieved great progress. In these work, local representations (LRs) are typically employed, because LRs are more consistent among the seen and unseen classes. However, most of them are limited to an individual image-to-image or image-to-class measure manner, which cannot fully exploit the capabilities of LRs, especially in the context of a certain task. This paper proposes an Adaptive Task-aware Local Representations Network (ATL-Net) to address this limitation by introducing episodic attention, which can adaptively select the important local patches among the entire task, as the process of human recognition. We achieve much superior results on multiple benchmarks. On the miniImagenet, ATL-Net gains 0.93% and 0.88% improvements over the compared methods under the 5-way 1-shot and 5-shot settings. Moreover, ATL-Net can naturally tackle the problem that how to adaptively identify and weight the importance of different key local parts, which is the major concern of fine-grained recognition. Specifically, on the fine-grained dataset Stanford Dogs, ATL-Net outperforms the second best method with 5.39% and 9.69% gains under the 5-way 1-shot and 5-shot settings.
BibTeX
@inproceedings{dong2020learning,
  author = {Chuanqi Dong and Wenbin Li and Jing Huo and Zheng Gu and Yang Gao},
  title = {{Learning Task-aware Local Representations for Few-shot Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/100},
}
Figure