NDPNet: A novel non-linear data projection network for few-shot fine-grained image classification

Weichuan Zhang, Xuefang Liu, Zhe Xue, Yongsheng Gao, Changming SunarXiv 2021 · arXiv

Abstract

Few-shot fine-grained image classification (FSFGIC) is a challenging task that requires distinguishing between highly similar sub-categories with extremely limited training data. This paper introduces NDPNet, a novel architecture that addresses these challenges through two key components: Feature Re-abstraction Embedding (FRaE) and Non-linear Data Projection (NDP). FRaE preserves fine-grained details by integrating multi-stage feature maps, while NDP enhances class separability by projecting descriptors into a non-linear space. Experimental results across multiple benchmarks show that NDPNet consistently outperforms existing state-of-the-art methods.

Introduction

The intersection of Few-Shot Learning (FSL) and Fine-Grained Image Classification (FGIC) presents a unique "double whammy." Standard FSL methods often struggle with high inter-class similarity, where subtle differences (e.g., the shape of a bird's beak or a car's headlight) are the only distinguishing factors. Conversely, FGIC typically requires large datasets to learn these nuances. Existing approaches often rely on high-level abstract features from final layers that discard low-level spatial details. Furthermore, linear similarity measures like cosine distance often fail to adequately separate closely related categories in the feature space.

Methodology

NDPNet is designed to capture fine-grained details and improve class discriminability through a two-module architecture that processes support and query images into local feature descriptors.

Overview of the NDPNet architecture, illustrating the flow from input images through the Feature Re-abstraction Embedding and Non-linear Data Projection modules to final similarity scores.

Feature Re-abstraction Embedding (FRaE)

Standard convolutional neural networks extract features hierarchically, where deeper layers capture semantic information but lose spatial resolution. To counteract this, the FRaE module re-combines feature maps from the middle and late stages of the network, upsamples them, and merges them back with high-resolution features from the initial stage. This "re-abstraction" ensures that the final feature descriptors contain both high-level category information and the fine-grained spatial details necessary for distinguishing similar classes.

Non-linear Data Projection (NDP)

The core innovation of NDPNet is the Non-linear Data Projection module. Instead of calculating similarity in the original linear feature space, NDPNet projects the descriptors of query and support images into a new non-linear space using the transformation: $$\hat{a} = \text{sign}(a)(1 - \exp(-\frac{a^2}{2\alpha^2}))$$ This projection is designed to "decouple" the relationship between different categories. By learning optimal projection factors ($\alpha$ and $\beta$), the model effectively shrinks the distance between samples of the same class while stretching the distance between samples of different classes, making the classification boundary much clearer.

Experimental Results

The model was evaluated on four popular fine-grained benchmarks: Stanford Dogs, Stanford Cars, CUB-200, and a specialized Cottons dataset. NDPNet consistently outperformed previous state-of-the-art (SOTA) methods across all tests.

Comparative performance of NDPNet against state-of-the-art models on the Stanford Dogs dataset for 5-way 1-shot and 5-shot classification tasks.

As shown in the results for the Stanford Dogs dataset, NDPNet achieves a 5-way 5-shot accuracy of 74.66%, significantly outperforming previous metric-learning methods like DN4 and CovaMNet. In the Stanford Cars dataset, NDPNet reached 71.48% accuracy in the 1-shot scenario, marking a substantial jump over the previous SOTA, ATL-Net (67.95%). These results demonstrate the model's robustness to extremely limited data.

Conclusion

NDPNet demonstrates that integrating low-level visual details with high-level semantics, combined with non-linear feature projection, is highly effective for few-shot fine-grained classification. While the model achieves state-of-the-art results, future work could address the computational overhead of the re-abstraction process and explore learning the optimal shape of the non-linear projection function through adaptive mechanisms like small multi-layer perceptrons.

Summary updated

Abstract

Metric-based few-shot fine-grained image classification (FSFGIC) aims to learn a transferable feature embedding network by estimating the similarities between query images and support classes from very few examples. In this work, we propose, for the first time, to introduce the non-linear data projection concept into the design of FSFGIC architecture in order to address the limited sample problem in few-shot learning and at the same time to increase the discriminability of the model for fine-grained image classification. Specifically, we first design a feature re-abstraction embedding network that has the ability to not only obtain the required semantic features for effective metric learning but also re-enhance such features with finer details from input images. Then the descriptors of the query images and the support classes are projected into different non-linear spaces in our proposed similarity metric learning network to learn discriminative projection factors. This design can effectively operate in the challenging and restricted condition of a FSFGIC task for making the distance between the samples within the same class smaller and the distance between samples from different classes larger and for reducing the coupling relationship between samples from different categories. Furthermore, a novel similarity measure based on the proposed non-linear data project is presented for evaluating the relationships of feature information between a query image and a support set. It is worth to note that our proposed architecture can be easily embedded into any episodic training mechanisms for end-to-end training from scratch. Extensive experiments on FSFGIC tasks demonstrate the superiority of the proposed methods over the state-of-the-art benchmarks.
BibTeX
@misc{zhang2021ndpnet,
  author = {Weichuan Zhang and Xuefang Liu and Zhe Xue and Yongsheng Gao and Changming Sun},
  title = {{NDPNet: A novel non-linear data projection network for few-shot fine-grained image classification}},
  year = {2021},
  eprint = {2106.06988},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.06988},
}
Figure