Discrimination-Aware Mechanism for Fine-Grained Representation Learning

Furong Xu, Meng Wang, Wei Zhang, Yuan Cheng, Wei ChuCVPR 2021 · DOI · OpenReview

TL;DR

This paper introduces a "Discrimination-Aware Mechanism" (DAM) that helps deep learning models learn more subtle visual cues for fine-grained recognition. By iteratively identifying and focusing on under-trained feature elements, DAM enhances the overall representation, leading to better generalization performance across various tasks like image classification and person re-identification.

What Problem Does This Paper Address?

Fine-grained recognition, the task of distinguishing between very similar sub-classes (e.g., different bird species, car models), is notoriously challenging. The differences between these sub-classes are often subtle and difficult for standard deep learning models to capture. Traditional training methods tend to optimize for overall feature discriminativeness, but they can prematurely focus on easily distinguishable features, neglecting the subtle cues that are crucial for accurate fine-grained classification. This leads to feature extractors that are not generalizable enough, performing poorly on unseen data or slightly different variations. The paper aims to address this by developing a mechanism that ensures all feature elements, including the subtle ones, are adequately trained to contribute to discriminative representations.

Methods

The core innovation is the Discrimination-Aware Mechanism (DAM). DAM works by iteratively identifying feature elements that are less discriminative across different classes. It then "gates" or selectively focuses on these less discriminative elements for further optimization, while temporarily downplaying or "erasing" the already highly discriminative ones. This process is guided by the differences between class centers (represented by the weights of the final fully connected layer).

Here's a breakdown:

  • Class Center Representation: The weights of the last fully connected layer are used as proxies for class centers. The difference between these weights for different classes indicates how discriminative a particular feature dimension is.
  • Gating Mechanism: Based on these inter-class differences, a gating mechanism is applied. Feature elements with low discriminativeness (i.e., small differences between class centers) are retained for further training, while those with high discriminativeness are temporarily masked. This forces the model to learn from the harder, subtler cues.
  • Iterative Refinement: This process is repeated iteratively. With each iteration, more feature elements are pushed to become discriminative, leading to a richer and more robust feature representation.
  • Integration with Existing Losses: DAM is designed to be plug-and-play. It can be seamlessly integrated with existing proxy-based losses (like Softmax Cross-Entropy) and pair-based losses (like Triplet Loss). For proxy-based losses, it retrains low-discriminative elements. For pair-based losses, it strategically uses high-discriminative elements for positive pairs and low-discriminative elements for negative pairs to increase learning difficulty and improve robustness.

Datasets

The paper evaluates its method on several standard datasets for fine-grained recognition and person re-identification:

  • CUB-200-2011: A dataset of bird species.
  • Cars196: A dataset of car models.
  • Market-1501: A person re-identification dataset.
  • MSMT17: Another person re-identification dataset.

The paper uses standard train/test splits for these datasets.

Key Insights and Ideas

  • Focusing on the "Hard" Features: The most significant insight is that traditional methods often overlook subtle discriminative cues because they get "stuck" optimizing easily learned features. DAM explicitly targets these harder-to-learn features, forcing the model to extract more comprehensive information.
  • Guided Erasing vs. Random Erasing: Unlike random erasing techniques that randomly remove parts of an image or feature map, DAM uses inter-class differences to guide which feature elements to focus on. This guided approach is more effective because it targets specific weaknesses in the feature representation.
  • Improving Generalization through Increased Difficulty: By making the learning process more challenging (by focusing on less discriminative features), DAM paradoxically improves the model's ability to generalize to new, unseen data. This is a common theme in deep learning where controlled increases in training difficulty can lead to better final performance.
  • Universality of the Mechanism: The fact that DAM can be easily integrated with both proxy-based and pair-based loss functions makes it a versatile tool applicable to a wide range of fine-grained recognition and retrieval tasks.

Key Results

The proposed DAM consistently improves performance across all tested datasets and baseline methods.

  • Fine-grained Classification (CUB-200-2011, Cars196): DAM, when applied to proxy-based (SCE) and pair-based (TriHard) losses, showed improvements. For example, SCEDAM achieved a +1.1% increase in Rank-1 accuracy on CUB-200-2011 and +1.2% on Cars196 compared to the baseline SCE. Combining DAM with both SCE and TriHard (DAM in Table 1) yielded even better results.
  • Person Re-Identification (Market-1501, MSMT17): DAM also demonstrated significant gains. On Market-1501, SCEDAM improved mAP by +0.9% and Rank-1 by +1.0% over SCE. On MSMT17, the gains were even more substantial, with SCEDAM improving mAP by +4.6% and Rank-1 by +4.7%.
  • State-of-the-Art Comparison: The paper shows that DAM-enhanced methods outperform several existing state-of-the-art approaches on these benchmarks, often achieving competitive or superior results with similar or smaller embedding sizes.

Limitations and Future Work

  • Parameter Tuning: The paper acknowledges that the effectiveness of DAM can be influenced by a parameter ($\lambda$) that controls the gating mechanism. While they found $\lambda=1.5$ to be optimal on Market-1501, this parameter might require tuning for different datasets or tasks.
  • Future Directions: The authors suggest exploring more sophisticated ways to select features for further optimization. They also express interest in generalizing DAM to other machine learning domains beyond fine-grained recognition.

Summary updated

Abstract

Recently, with the emergence of retrieval requirements for certain individual in the same superclass, e.g., birds, persons, cars, fine-grained recognition task has attracted a significant amount of attention from academia and industry. In fine-grained recognition scenario, the inter-class differences are quite diverse and subtle, which makes it challenging to extract all the discriminative cues. Traditional training mechanism optimizes the overall discriminativeness of the whole feature. It may stop early when some feature elements has been trained to distinguish training samples well, leaving other elements insufficiently trained for a feature. This would result in a less generalizable feature extractor that only captures major discriminative cues and ignores subtle ones. Therefore, there is a need for a training mechanism that enforces the discriminativeness of all the elements in the feature to capture more the subtle visual cues. In this paper, we propose a Discrimination-Aware Mechanism (DAM) that iteratively identifies insufficiently trained elements and improves them. DAM is able to increase the number of well learned elements, which captures more visual cues by the feature extractor. In this way, a more informative representation is learned, which brings better generalization performance. We show that DAM can be easily applied to both proxy-based and pair-based loss functions, and thus can be used in most existing fine-grained recognition paradigms. Comprehensive experiments on CUB-200-2011, Cars196, Market-1501, and MSMT17 datasets demonstrate the advantages of our DAM based loss over the related state-of-the-art approaches.
BibTeX
@inproceedings{xu2021discrimination,
  author = {Furong Xu and Meng Wang and Wei Zhang and Yuan Cheng and Wei Chu},
  title = {{Discrimination-Aware Mechanism for Fine-Grained Representation Learning}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  doi = {10.1109/cvpr46437.2021.00087},
}
Figure