Sharp Attention Network via Adaptive Sampling for Person Re-Identification

Chen Shen, Guo-Jun Qi, Rongxin Jiang, Zhongming Jin, Hongwei Yong, Yaowu Chen, Xian-Sheng Hua

This paper, published in IEEE Transactions on Circuits and Systems for Video Technology, presents a shift in how deep learning models "pay attention" to images, specifically for the task of person re-identification (re-ID).

TL;DR

The authors introduce a Sharp Attention Network that replaces traditional "soft" gating (which often produces blurry, ambiguous attention masks) with a differentiable sampling mechanism. By using the Gumbel-Softmax trick to "sharpen" feature selection, the model more assertively localizes subtle, discriminative details—like a specific backpack strap or shoe color—while effectively trimming out background noise.

What Problem Does This Paper Address?

In person re-identification, a system must match the same individual across different non-overlapping camera views. This is incredibly difficult because of changes in lighting, pose, and background clutter.

Most current "attention" mechanisms in deep learning use soft gating (typically a Sigmoid function). This produces a continuous mask where values range from 0 to 1. The problem? These masks are often "blurry," with many values hovering around 0.5. This ambiguity means the model isn't quite sure whether a feature is important or not, leading it to retain irrelevant background noise or miss subtle visual cues that are essential for distinguishing between two similar-looking people.

Methods

The core innovation is the Sharp Attention Mask Generator (SAMG). Instead of just scaling features by a soft decimal, the authors treat attention as a binary selection problem: Should I use this pixel or not?

  1. Adaptive Sampling via Gumbel-Softmax: To make the discrete "yes/no" sampling process differentiable (and thus trainable via backpropagation), the authors employ the Gumbel-Softmax trick. This allows the network to approximate Bernoulli sampling, producing "sharp" masks that are nearly 0 or 1.
  2. Context-Aware Unit: Before sampling, a U-Net-like structure (encoder-decoder) captures high-level context from a large receptive field. This ensures the "sharp" decision is informed by the surrounding scene, helping the model ignore background clutter in poorly cropped images.
  3. Total Variation (TV) Regularization: Because random sampling can introduce "salt-and-pepper" noise into the masks, the authors add a TV regularization term to the loss function. This encourages the attention masks to be spatially continuous and smooth.
  4. Sharp Attention Blocks: These blocks are integrated into a standard ResNet-50 backbone, allowing for hierarchical attention refinement from coarse to fine levels.

Datasets

The model was evaluated on three gold-standard benchmarks for person re-ID:

  • CUHK03: Includes both manually labeled and automatically detected (noisier) bounding boxes.
  • Market-1501: A large-scale dataset with over 32,000 images and 1,501 identities.
  • DukeMTMC-reID: A challenging dataset derived from a multi-camera tracking system.

Key Insights and Ideas

  • Assertiveness is Key: The "aha moment" is that for fine-grained tasks like re-ID, being "unsure" (soft attention) is worse than being "wrong" occasionally. Forcing the model to commit to specific features prevents it from overfitting to the "average" background.
  • Trimming the Fat: By enforcing a "parsimony prior," the sharp attention mechanism effectively "trims" irrelevant features. This makes the resulting feature embeddings much more robust.
  • Efficiency: The authors show that their sharp attention mechanism requires significantly fewer parameters (about 25% of the parameters used in previous soft attention models like the Residual Attention Network) while achieving better results.

Key Results

The Sharp Attention Network consistently outperformed the baseline ResNet-50 and other state-of-the-art methods:

  • CUHK03 (Labeled): Achieved 88.3% Rank-1 accuracy, a significant jump from the 85.1% baseline.
  • Market-1501: Reached 85.9% Rank-1 and 70.1% mAP (mean Average Precision).
  • DukeMTMC-reID: Hit 77.9% Rank-1, outperforming existing methods like ACRN and SVDNet.
  • Qualitative Success: Visualization of the masks shows that while soft attention often highlights the entire person and some background, sharp attention tightly "locks on" to specific discriminative regions like bags, logos, or unique clothing patterns.

Limitations and Future Work

While the sampling-based approach is powerful, it introduces randomness during the training phase. The authors mitigate this with TV regularization and temperature annealing for the Gumbel-Softmax, but this adds hyperparameter complexity.

Future work could explore applying this "sharp" philosophy to other domains where subtle feature localization is critical, such as medical imaging or fine-grained species classification. The authors also suggest that further optimizing the context-aware unit could yield even better results in extremely crowded or occluded environments.

Summary updated

Abstract

In this paper, we present novel sharp attention networks by adaptively sampling feature maps from convolutional neural networks for person re-identification (re-ID) problems. Due to the introduction of sampling-based attention models, the proposed approach can adaptively generate sharper attention-aware feature masks. This greatly differs from the gating-based attention mechanism that relies on soft gating functions to select the relevant features for person re-ID. In contrast, the proposed sampling-based attention mechanism allows us to effectively trim irrelevant features by enforcing the resultant feature masks to focus on the most discriminative features. It can produce sharper attentions that is more assertive in localizing subtle features relevant to re-identifying people across cameras. For this purpose, a differentiable Gumbel-Softmax sampler is employed to approximate the Bernoulli sampling to train the sharp attention networks. Extensive experimental evaluations demonstrate the superiority of this new sharp attention model for person re-ID over other related existing, published state-of-the-art works on three challenging benchmarks, including CUHK03, Market-1501, and DukeMTMC-reID.
BibTeX
@article{shen2019sharp,
  author = {Chen Shen and Guo-Jun Qi and Rongxin Jiang and Zhongming Jin and Hongwei Yong and Yaowu Chen and Xian-Sheng Hua},
  title = {{Sharp Attention Network via Adaptive Sampling for Person Re-Identification}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2019},
  doi = {10.1109/tcsvt.2018.2872503},
}
Figure