Exploration of Class Center for Fine-Grained Visual Classification

Hang Yao, Qiguang Miao, Peipei Zhao, Chaoneng Li, Xin Li, Guanwen Feng, Ruyi Liu

Abstract

Different from large-scale classification tasks, fine-grained visual classification is a challenging task due to two critical problems: 1) evident intra-class variances and subtle inter-class differences, and 2) overfitting owing to fewer training samples in datasets. Most existing methods extract key features to reduce intra-class variances, but pay no attention to subtle inter-class differences in fine-grained visual classification. To address this issue, we propose a loss function named exploration of class center (ECC), which consists of a multiple class-center constraint and a class-center label generation. This loss function fully utilizes the information of the class center from the perspective of features and labels. From the feature perspective, the multiple class-center constraint pulls samples closer to the target class center, and pushes samples away from the most similar nontarget class center. Thus, the constraint reduces intra-class variances and enlarges inter-class differences. From the label perspective, the class-center label generation utilizes class-center distributions to generate soft labels to alleviate overfitting. Our method can be easily integrated with existing fine-grained visual classification approaches as a loss function, to further boost excellent performance with only slight training costs. Extensive experiments are conducted to demonstrate consistent improvements achieved by our method on four widely-used fine-grained visual classification datasets. In particular, our method achieves state-of-the-art performance on the FGVC-Aircraft and CUB-200-2011 datasets.

Introduction

Fine-Grained Visual Classification (FGVC) represents a specialized domain of computer vision where models must distinguish between highly similar sub-categories, such as specific bird species or aircraft models. This task is significantly more difficult than general classification due to high intra-class variance (the same species appearing different in various poses) and subtle inter-class differences (different species appearing nearly identical). Furthermore, FGVC datasets are typically small, leading to a high risk of overfitting. While many existing approaches focus on identifying discriminative "key parts," they often neglect the subtle boundaries between visually similar classes or suffer from training instabilities when defining class centers.

The Exploration of Class Center (ECC) Framework

To overcome these challenges, the Exploration of Class Center (ECC) framework introduces two primary components designed to refine feature representations and label supervision.

The ECC framework overview, illustrating the Multiple Class-Center Constraint (MCC) and Class-Center Label Generation (CLG) components.

Multiple Class-Center Constraint (MCC)

Operating in the feature space, the MCC goes beyond traditional center loss. While standard center loss only minimizes the distance between a sample and its own class center, MCC actively pushes the sample away from the most similar non-target class center. This "push-pull" mechanism specifically addresses the "look-alike" problem by sharpening decision boundaries between the most confusing categories.

Class-Center Label Generation (CLG)

At the label level, CLG addresses overfitting by moving away from "hard" one-hot labels. Instead of assuming 100% certainty or using uniform label smoothing, CLG generates soft labels based on the actual distribution of class centers. If two classes are visually similar, the soft label for an image will reflect this relationship, providing a more realistic and nuanced supervision signal that prevents overconfidence.

A key technical contribution is the stable update strategy for class centers. Rather than treating centers as learnable parameters updated via backpropagation—which can be unstable—the authors use a running average of features. This ensures smoother convergence and more robust feature learning.

Experimental Results

The ECC method was evaluated on four major FGVC benchmarks: CUB-200-2011 (birds), FGVC-Aircraft (AIR), Stanford Cars (CAR), and NABirds (NAB).

t-SNE visualization of feature distributions comparing standard losses with the proposed ECC method, showing tighter intra-class clusters and clearer inter-class separation.

Performance and Efficiency

The method achieved state-of-the-art results on FGVC-Aircraft (95.2%) and CUB-200-2011 (92.3%). As a "plug-and-play" loss function, ECC demonstrated consistent improvements across various backbones, including ResNet50, DenseNet121, and Swin Transformers. Notably, the computational overhead is minimal and only applies during training, making it highly efficient for deployment.

Key Insights

  • Targeted Separation: Focusing the discriminative force on the "nearest neighbor" non-target class is more effective for FGVC than general inter-class separation.
  • Realistic Supervision: Soft labels derived from class center distributions effectively regularize the model, acknowledging inherent visual similarities between species.

Conclusion

The Exploration of Class Center framework provides a robust and efficient solution for the dual challenges of intra-class variance and inter-class similarity in FGVC. By refining both feature space constraints and label supervision, ECC achieves superior performance with negligible extra cost. Future work may involve integrating these class-center insights with advanced part-based attention mechanisms to further enhance discriminative power.

Summary updated

Abstract

Different from large-scale classification tasks, fine-grained visual classification is a challenging task due to two critical problems: 1) evident intra-class variances and subtle inter-class differences, and 2) overfitting owing to fewer training samples in datasets. Most existing methods extract key features to reduce intra-class variances, but pay no attention to subtle inter-class differences in fine-grained visual classification. To address this issue, we propose a loss function named exploration of class center, which consists of a multiple class-center constraint and a class-center label generation. This loss function fully utilizes the information of the class center from the perspective of features and labels. From the feature perspective, the multiple class-center constraint pulls samples closer to the target class center, and pushes samples away from the most similar nontarget class center. Thus, the constraint reduces intra-class variances and enlarges inter-class differences. From the label perspective, the class-center label generation utilizes classcenter distributions to generate soft labels to alleviate overfitting. Our method can be easily integrated with existing fine-grained visual classification approaches as a loss function, to further boost excellent performance with only slight training costs. Extensive experiments are conducted to demonstrate consistent improvements achieved by our method on four widely-used fine-grained visual classification datasets. In particular, our method achieves state-of-the-art performance on the FGVC-Aircraft and CUB-200-2011 datasets.
BibTeX
@article{yao2024exploration,
  author = {Hang Yao and Qiguang Miao and Peipei Zhao and Chaoneng Li and Xin Li and Guanwen Feng and Ruyi Liu},
  title = {{Exploration of Class Center for Fine-Grained Visual Classification}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2024},
  url = {https://openreview.net/forum?id=tmTmXqWJTX},
}
Figure