Abstract
Few-shot fine-grained image classification (FSFGIC) is a challenging task that requires distinguishing between highly similar sub-categories with extremely limited training data. This paper introduces NDPNet, a novel architecture that addresses these challenges through two key components: Feature Re-abstraction Embedding (FRaE) and Non-linear Data Projection (NDP). FRaE preserves fine-grained details by integrating multi-stage feature maps, while NDP enhances class separability by projecting descriptors into a non-linear space. Experimental results across multiple benchmarks show that NDPNet consistently outperforms existing state-of-the-art methods.
Introduction
The intersection of Few-Shot Learning (FSL) and Fine-Grained Image Classification (FGIC) presents a unique "double whammy." Standard FSL methods often struggle with high inter-class similarity, where subtle differences (e.g., the shape of a bird's beak or a car's headlight) are the only distinguishing factors. Conversely, FGIC typically requires large datasets to learn these nuances. Existing approaches often rely on high-level abstract features from final layers that discard low-level spatial details. Furthermore, linear similarity measures like cosine distance often fail to adequately separate closely related categories in the feature space.
Methodology
NDPNet is designed to capture fine-grained details and improve class discriminability through a two-module architecture that processes support and query images into local feature descriptors.
Feature Re-abstraction Embedding (FRaE)
Standard convolutional neural networks extract features hierarchically, where deeper layers capture semantic information but lose spatial resolution. To counteract this, the FRaE module re-combines feature maps from the middle and late stages of the network, upsamples them, and merges them back with high-resolution features from the initial stage. This "re-abstraction" ensures that the final feature descriptors contain both high-level category information and the fine-grained spatial details necessary for distinguishing similar classes.
Non-linear Data Projection (NDP)
The core innovation of NDPNet is the Non-linear Data Projection module. Instead of calculating similarity in the original linear feature space, NDPNet projects the descriptors of query and support images into a new non-linear space using the transformation: $$\hat{a} = \text{sign}(a)(1 - \exp(-\frac{a^2}{2\alpha^2}))$$ This projection is designed to "decouple" the relationship between different categories. By learning optimal projection factors ($\alpha$ and $\beta$), the model effectively shrinks the distance between samples of the same class while stretching the distance between samples of different classes, making the classification boundary much clearer.
Experimental Results
The model was evaluated on four popular fine-grained benchmarks: Stanford Dogs, Stanford Cars, CUB-200, and a specialized Cottons dataset. NDPNet consistently outperformed previous state-of-the-art (SOTA) methods across all tests.
As shown in the results for the Stanford Dogs dataset, NDPNet achieves a 5-way 5-shot accuracy of 74.66%, significantly outperforming previous metric-learning methods like DN4 and CovaMNet. In the Stanford Cars dataset, NDPNet reached 71.48% accuracy in the 1-shot scenario, marking a substantial jump over the previous SOTA, ATL-Net (67.95%). These results demonstrate the model's robustness to extremely limited data.
Conclusion
NDPNet demonstrates that integrating low-level visual details with high-level semantics, combined with non-linear feature projection, is highly effective for few-shot fine-grained classification. While the model achieves state-of-the-art results, future work could address the computational overhead of the re-abstraction process and explore learning the optimal shape of the non-linear projection function through adaptive mechanisms like small multi-layer perceptrons.
Summary updated