Abstract
Few-shot learning (FSL) aims to recognize novel categories from minimal labeled examples. Traditional metric-learning approaches often rely on global image features, which can be sensitive to background noise and intra-class variations. We present the Augmented Bi-path Network (ABNet), which integrates a Global Path (GP) and a Local Path (LP) to capture both holistic and salient part-based features. ABNet further employs a learnable feature augmentation module and an attention-based re-weighting mechanism to enhance robustness against viewpoint and scale changes. Experimental results on miniImageNet, tieredImageNet, and Caltech-256 demonstrate that ABNet achieves state-of-the-art performance, significantly outperforming existing benchmarks.
Introduction: The Challenge of Few-Shot Learning
Deep learning typically requires massive amounts of labeled data. Few-shot Learning (FSL) aims to bridge the gap between machine and human intelligence by training models to recognize new categories from just one or five examples.
Existing FSL methods often struggle because they rely on "image-level" comparisons. They look at the whole picture at once, which can lead to mistakes. For instance, a bird and a dog might look globally similar due to background or lighting, but their local features (beaks vs. paws) are distinct. Furthermore, with so few training samples, models struggle to account for intra-class variation—the fact that the same object can look very different depending on the camera angle, scale, or orientation.
The ABNet Architecture
The authors propose ABNet, which introduces three major innovations over standard "learn-to-compare" architectures:
- Bi-path Comparison (Global + Local): Instead of just looking at the whole image, the model extracts "salient patches" (informative parts like heads or legs). It processes the whole image in a Global Path (GP) and the patches in a Local Path (LP).
- Learning to Augment (LA): Rather than relying solely on manual data augmentation (like random flips), ABNet includes a module that learns a set of affine transformation matrices. These matrices transform feature maps to simulate different viewpoints and scales, making the model more robust to how an object might appear in a query image.
- Learning to Compare & Re-weight: In the Local Path, not every patch is useful (some might just be background). The model uses a learnable attention module to calculate a weight for each patch pair, emphasizing class-relevant features and suppressing noise.
The salient patches are extracted offline using a combination of Selective Search (to find object-like regions) and Minimum Barrier Distance (to measure visual salience against the background).
Experimental Evaluation
The model was evaluated on three widely recognized FSL benchmarks: miniImageNet, Caltech-256, and tieredImageNet. ABNet achieved state-of-the-art results across all tested datasets using a standard 4-layer backbone:
- miniImageNet: Reached 58.12% accuracy for 1-shot tasks, a significant jump over the previous best of ~52.8%.
- tieredImageNet: Achieved 62.10% for 1-shot tasks, outperforming the runner-up by over 7%.
- Ablation Success: The authors proved that every piece of the puzzle matters. Adding the "Learning to Augment" module improved 1-shot accuracy by ~1.8%, and adding the "Learning to Re-weight" module added another ~2.3%.
Discussion and Key Insights
The paper demonstrates that while global features might suggest two images are similar, local patches provide the "ground truth" for classification. This mimics how humans often identify objects by looking for specific parts. By learning to augment features directly rather than just raw pixels, the model can more effectively "hallucinate" what a support image would look like from a different perspective. Furthermore, the learnable re-weighting module is crucial because it allows the model to ignore irrelevant similarities (e.g., background noise).
Limitations and Future Work
While effective, the current ABNet pipeline relies on offline salient patch extraction. Integrating this into an end-to-end trainable "glimpse" network could improve efficiency. Additionally, the model currently uses a fixed number of patches, which may not be optimal for all object types. Future research could explore more complex non-linear transformations for augmentation beyond affine matrices to capture even more diverse intra-class variations.
Summary updated