Grad-Cam Guided Progressive Feature CutMix for Classification

Yan Zhang, Binyu He, Li SunarXiv 2020 · OpenReview

TL;DR

This paper introduces a novel "Progressive Multi-stage Feature Mix" (PMM) network for person re-identification. By progressively focusing on different image regions and strategically mixing features from negative examples, PMM encourages models to learn more robust and diverse representations, leading to significant performance improvements.

What Problem Does This Paper Address?

Person re-identification (reID) aims to match individuals across different camera views, a task crucial for surveillance and security. A major challenge is that Convolutional Neural Networks (CNNs) often over-focus on the most visually salient parts of an image (e.g., a person's face or torso), potentially missing other discriminative clues like clothing details, shoes, or accessories. This can lead to misidentification, especially when dealing with occlusions, viewpoint changes, or similar-looking individuals. Existing methods like "Batch DropBlock" (BDB) attempt to address this by randomly dropping feature blocks, but this can lead to the model attending to irrelevant regions.

Methods

The core innovation is the Progressive Multi-stage Feature Mix (PMM) network. It consists of three sequential "heads" or stages, each building upon the previous one.

  • Progressive Multi-Branch Structure: Instead of a single classifier, PMM uses multiple stages. The first stage receives the raw features from the backbone (ResNet50). Subsequent stages receive features that have been "mixed" based on insights from the preceding stage. This progressive approach forces the model to look for different, less obvious clues in each stage.
  • Attentive Hard-Mix (A-Hard-Mix): This is the key feature augmentation technique. For each stage, it uses Grad-CAM (Gradient-weighted Class Activation Mapping) to identify the most salient regions in the current feature map. These salient regions are then replaced with corresponding regions from a negative example (an image of a different person). This creates a "harder" training sample, forcing the model to learn from less obvious features and preventing it from relying solely on the most prominent ones. The process involves:
  • Block Ranking: Grad-CAM identifies important regions. These regions are ranked, and the top-ranked (most salient) blocks are masked out.
  • Feature Hard-Mix: The masked-out regions in the "anchor" feature map are replaced by features from the same spatial locations in a "negative" feature map.

Datasets

The proposed method was evaluated on three standard person re-identification benchmarks:

  • Market-1501
  • DukeMTMC-reID
  • CUHK03-Detected

Key Insights and Ideas

  • Progressive Feature Exploration: The staged approach is crucial. By forcing the model to look for new clues after suppressing the most obvious ones, PMM encourages a more comprehensive understanding of a person's appearance.
  • Strategic Feature Mixing: Replacing salient features with those from negative examples is more effective than random dropping. This "hard negative" mixing creates challenging training samples that push the model to learn finer-grained distinctions.
  • Grad-CAM as a Guide: Using Grad-CAM to identify salient regions provides a principled way to guide the feature mixing process, ensuring that the most informative parts of the current stage are targeted for replacement.

Key Results

The PMM network achieved significant performance improvements across all tested datasets. For instance, on the challenging CUHK03 dataset, the proposed method showed a notable gain compared to the BDB baseline. The paper reports Rank-1 accuracy and mAP scores, with the proposed method outperforming state-of-the-art approaches. For example, on CUHK03, the proposed method achieved 76.3% Rank-1 accuracy and 73.0% mAP, outperforming BDB* (73.5% Rank-1, 69.8% mAP).

Limitations and Future Work

  • Training Complexity: The progressive nature and the sequential back-propagation required for Grad-CAM computation in each stage increase the training time compared to single-stage methods.
  • Testing Phase Simplification: During testing, the A-Hard-Mix operation is discarded as the guidance from Grad-CAM is not readily available. The final representation is formed by concatenating features from all stages.
  • Future Work: The authors suggest exploring further ways to enhance feature diversity and robustness, potentially by investigating different mixing strategies or more advanced attention mechanisms.

Summary updated

Abstract

Image features from a small local region often give strong evidence in person re-identification task. However, CNN suffers from paying too much attention on the most salient local areas, thus ignoring other discriminative clues, e.g., hair, shoes or logos on clothes. %BDB proposes to randomly drop one block in a batch to enlarge the high response areas. Although BDB has achieved remarkable results, there still room for improvement. In this work, we propose a Progressive Multi-stage feature Mix network (PMM), which enables the model to find out the more precise and diverse features in a progressive manner. Specifically, 1. to enforce the model to look for different clues in the image, we adopt a multi-stage classifier and expect that the model is able to focus on a complementary region in each stage. 2. we propose an Attentive feature Hard-Mix (A-Hard-Mix) to replace the salient feature blocks by the negative example in the current batch, whose label is different from the current sample. 3. extensive experiments have been carried out on reID datasets such as the Market-1501, DukeMTMC-reID and CUHK03, showing that the proposed method can boost the re-identification performance significantly.
BibTeX
@misc{zhang2020grad,
  author = {Yan Zhang and Binyu He and Li Sun},
  title = {{Grad-Cam Guided Progressive Feature CutMix for Classification}},
  year = {2020},
  eprint = {2007.08779},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2007.08779},
}
Figure