Few-Shot Fine-Grained Image Classification with Progressively Feature Refinement and Continuous Relationship Modeling

Zhen-Xiang Ma, Zhen-Duo Chen, Tai Zheng, Xin Luo, Zixia Jia, Xin-Shun Xu

Abstract

Few-shot fine-grained image classification (FGIC) aims to categorize visually similar objects using only a few labeled examples. This task is particularly difficult because standard deep learning models often overfit to noise or fail to capture the subtle discriminative features necessary for fine-grained distinction. This paper presents the SUITED framework (progreSsively featUre refInement and conTinuous rElationship moDeling), which addresses these challenges through two primary innovations: a top-down feature refinement mechanism and a graph-based relationship modeling module. Evaluated across five major benchmarks, SUITED achieves state-of-the-art results, demonstrating superior ability in feature representation and class separation.

Introduction

Fine-grained image classification (FGIC) involves distinguishing between sub-categories with high visual similarity, such as specific bird species or car models. When combined with Few-Shot Learning (FSL), where only one or five examples per class are available, the task becomes significantly more complex. Current methodologies face two primary hurdles: (1) backbones often capture irrelevant background noise instead of discriminative details, and (2) relationship modeling typically relies on discrete, one-to-one comparisons between a query and a single prototype, ignoring the global feature distribution.

Methodology

The SUITED framework introduces two novel modules to overcome these limitations:

  1. Progressive Feature Refinement Module (PFRM): PFRM implements a top-down feedback mechanism. High-level semantic features are used to guide the refinement of lower-level feature maps. This ensures that the model focuses on category-discriminative regions rather than background noise, creating a robust multi-scale representation.
  2. Continuous Relationship Modeling Module (CRMM): Unlike traditional discrete modeling, CRMM adopts a "one-to-many" approach. By treating prototypes and query features as nodes in a Graph Convolutional Network (GCN), the model considers the relationship between a query and all class prototypes simultaneously. This global perspective allows for more precise optimization of the query's position in the feature space, maintaining distance from similar "distractor" classes.
The SUITED framework architecture, illustrating the integration of the Progressive Feature Refinement Module (PFRM) and the Continuous Relationship Modeling Module (CRMM) within the backbone network.

Experiments and Results

The authors evaluated SUITED on five standard benchmarks: CUB-200-2011, Stanford Dogs, Stanford Cars, meta-iNat, and tiered meta-iNat. SUITED consistently outperformed existing state-of-the-art methods. On the CUB-200-2011 dataset, it achieved 86.02% accuracy in the 1-shot setting and 94.13% in the 5-shot setting using a ResNet-12 backbone. On the large-scale meta-iNat dataset, it maintained a lead of approximately 3% over previous methods.

Qualitative analysis via t-SNE visualizations confirms that the combination of PFRM and CRMM leads to significantly tighter clusters and clearer decision boundaries compared to baseline models.

t-SNE visualizations of feature distributions for (a) Base model, (b) Base+PFRM, (c) Base+CRMM, and (d) the full SUITED model (Base+PFRM+CRMM), demonstrating improved class separation.

Conclusion

The SUITED framework sets a new benchmark for few-shot fine-grained image classification by refining feature extraction through top-down guidance and improving relationship modeling via a global, continuous approach. While the training phase involves additional computational complexity due to the GCN and feedback loops, the CRMM can be removed during inference without significant performance loss. Future work may explore the application of these techniques to more complex "in-the-wild" scenarios and other few-shot tasks.

Summary updated

BibTeX
@inproceedings{ma2025few,
  author = {Zhen-Xiang Ma and Zhen-Duo Chen and Tai Zheng and Xin Luo and Zixia Jia and Xin-Shun Xu},
  title = {{Few-Shot Fine-Grained Image Classification with Progressively Feature Refinement and Continuous Relationship Modeling}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i6.32645},
}
Figure