Augmented Bi-path Network for Few-shot Learning

Baoming Yan, Chen Zhou, Bo Zhao, Kan Guo, Jiang Yang, Xiaobo Li, Ming Zhang, Yizhou Wang

Abstract

Few-shot learning (FSL) aims to recognize novel categories from minimal labeled examples. Traditional metric-learning approaches often rely on global image features, which can be sensitive to background noise and intra-class variations. We present the Augmented Bi-path Network (ABNet), which integrates a Global Path (GP) and a Local Path (LP) to capture both holistic and salient part-based features. ABNet further employs a learnable feature augmentation module and an attention-based re-weighting mechanism to enhance robustness against viewpoint and scale changes. Experimental results on miniImageNet, tieredImageNet, and Caltech-256 demonstrate that ABNet achieves state-of-the-art performance, significantly outperforming existing benchmarks.

Introduction: The Challenge of Few-Shot Learning

Deep learning typically requires massive amounts of labeled data. Few-shot Learning (FSL) aims to bridge the gap between machine and human intelligence by training models to recognize new categories from just one or five examples.

Existing FSL methods often struggle because they rely on "image-level" comparisons. They look at the whole picture at once, which can lead to mistakes. For instance, a bird and a dog might look globally similar due to background or lighting, but their local features (beaks vs. paws) are distinct. Furthermore, with so few training samples, models struggle to account for intra-class variation—the fact that the same object can look very different depending on the camera angle, scale, or orientation.

Conceptual comparison between standard FSL methods (a) and ABNet (b), highlighting the use of local patches and augmentation to correct classification errors.

The ABNet Architecture

The authors propose ABNet, which introduces three major innovations over standard "learn-to-compare" architectures:

  1. Bi-path Comparison (Global + Local): Instead of just looking at the whole image, the model extracts "salient patches" (informative parts like heads or legs). It processes the whole image in a Global Path (GP) and the patches in a Local Path (LP).
  2. Learning to Augment (LA): Rather than relying solely on manual data augmentation (like random flips), ABNet includes a module that learns a set of affine transformation matrices. These matrices transform feature maps to simulate different viewpoints and scales, making the model more robust to how an object might appear in a query image.
  3. Learning to Compare & Re-weight: In the Local Path, not every patch is useful (some might just be background). The model uses a learnable attention module to calculate a weight for each patch pair, emphasizing class-relevant features and suppressing noise.
Detailed architecture of ABNet showing the Global Path (GP), Local Path (LP), and the Learning to Augment module.

The salient patches are extracted offline using a combination of Selective Search (to find object-like regions) and Minimum Barrier Distance (to measure visual salience against the background).

Experimental Evaluation

The model was evaluated on three widely recognized FSL benchmarks: miniImageNet, Caltech-256, and tieredImageNet. ABNet achieved state-of-the-art results across all tested datasets using a standard 4-layer backbone:

  • miniImageNet: Reached 58.12% accuracy for 1-shot tasks, a significant jump over the previous best of ~52.8%.
  • tieredImageNet: Achieved 62.10% for 1-shot tasks, outperforming the runner-up by over 7%.
  • Ablation Success: The authors proved that every piece of the puzzle matters. Adding the "Learning to Augment" module improved 1-shot accuracy by ~1.8%, and adding the "Learning to Re-weight" module added another ~2.3%.

Discussion and Key Insights

The paper demonstrates that while global features might suggest two images are similar, local patches provide the "ground truth" for classification. This mimics how humans often identify objects by looking for specific parts. By learning to augment features directly rather than just raw pixels, the model can more effectively "hallucinate" what a support image would look like from a different perspective. Furthermore, the learnable re-weighting module is crucial because it allows the model to ignore irrelevant similarities (e.g., background noise).

Limitations and Future Work

While effective, the current ABNet pipeline relies on offline salient patch extraction. Integrating this into an end-to-end trainable "glimpse" network could improve efficiency. Additionally, the model currently uses a fixed number of patches, which may not be optimal for all object types. Future research could explore more complex non-linear transformations for augmentation beyond affine matrices to capture even more diverse intra-class variations.

Summary updated

Abstract

Few-shot Learning (FSL) which aims to learn from few labeled training data is becoming a popular research topic, due to the expensive labeling cost in many real-world applications. One kind of successful FSL method learns to compare the testing (query) image and training (support) image by simply concatenating the features of two images and feeding it into the neural network. However, with few labeled data in each class, the neural network has difficulty in learning or comparing the local features of two images. Such simple image-level comparison may cause serious mis-classification. To solve this problem, we propose Augmented Bi-path Network (ABNet) for learning to compare both global and local features on multi-scales. Specifically, the salient patches are extracted and embedded as the local features for every image. Then, the model learns to augment the features for better robustness. Finally, the model learns to compare global and local features separately, i.e., in two paths, before merging the similarities. Extensive experiments show that the proposed ABNet outperforms the state-of-the-art methods. Both quantitative and visual ablation studies are provided to verify that the proposed modules lead to more precise comparison results.
BibTeX
@inproceedings{yan2020augmented,
  author = {Baoming Yan and Chen Zhou and Bo Zhao and Kan Guo and Jiang Yang and Xiaobo Li and Ming Zhang and Yizhou Wang},
  title = {{Augmented Bi-path Network for Few-shot Learning}},
  booktitle = {ICPR},
  year = {2020},
  url = {https://openreview.net/forum?id=dL3fCclKzrC},
}
Figure