Black Re-ID: A Head-shoulder Descriptor for the Challenging Problem of Person Re-Identification

Boqiang Xu, Lingxiao He, Xingyu Liao, Wu Liu, Zhenan Sun, Tao Mei

This paper, presented at ACM Multimedia 2020, tackles a specific but pervasive "blind spot" in computer vision: identifying people when their most distinctive feature—their clothing—is uninformative.

TL;DR

The authors introduce the Black Re-ID problem, where traditional person re-identification (Re-ID) fails because subjects wear black clothing or are captured in low light. They propose a Head-shoulder Adaptive Attention (HAA) network that uses a lightweight localization layer to extract "attribute-rich" features from the head and shoulders, dynamically weighting these features when clothing details are missing.

What Problem Does This Paper Address?

Most modern Person Re-ID systems are "clothing-centric." They achieve high accuracy by learning the colors, textures, and patterns of a person's outfit. However, in real-world surveillance, two major issues arise:

  1. The "Uniform" Problem: Many people wear black or dark clothing, making them look nearly identical to a standard algorithm.
  2. Low Illumination: In poor lighting, even colorful clothes appear black, leading to a massive loss of discriminative data.

The authors call this the Black Re-ID problem. When clothing attributes vanish, the system must rely on more subtle cues like hairstyles, head shape, glasses, or facial features—all of which are concentrated in the head-shoulder region.

Methods

The authors propose a two-stream architecture called the Head-shoulder Adaptive Attention (HAA) network.

  • Two-Stream Feature Extraction: The model consists of a Global Stream (extracting standard full-body features) and a Head-shoulder Attention Stream (HSA).
  • Head-shoulder Localization Layer (HLL): Instead of using heavy, pre-trained pose estimators (which make models slow), they designed a lightweight HLL inspired by Spatial Transformer Networks (STN). It learns to automatically "crop and zoom" into the head-shoulder region during training.
  • Head-shoulder Attention Network (HAN): This module slices the head-shoulder region into horizontal grids and applies both channel and spatial attention to find the most discriminative "micro-features" (like the presence of a crew cut or glasses).
  • Adaptive Attention Module: This is the "brain" of the ensemble. It first predicts if a person is wearing black. If they are, the module automatically assigns a higher weight to the head-shoulder features relative to the global features. If the person is wearing colorful clothes, it relies more on the global stream.

Datasets

The authors contributed a new benchmark to the community:

  • Black-reID Dataset: The first of its kind, containing 1,274 identities. It is divided into a "Black Group" (focusing on the core challenge) and a "White Group" (to test if the model works for other uniform-like colors).
  • Standard Benchmarks: They also evaluated on Market1501 and DukeMTMC-reID to ensure the model remains robust for general Re-ID tasks.

Key Insights and Ideas

  • The Head-Shoulder Region is a Goldmine: While clothing is volatile, the head-shoulder region contains stable, high-entropy information (haircut, gender cues, accessories) that remains visible even when the rest of the body is a black silhouette.
  • Pose Estimation is Overkill: The paper demonstrates that you don't need a complex skeleton-tracking model to find a head. A simple, end-to-end trainable localization layer (HLL) is sufficient and much more efficient for real-time applications.
  • Dynamic Weighting vs. Concatenation: Most models simply "stack" local and global features. This paper argues that the importance of a feature depends on the input. If the global view is a black blob, the local view must be prioritized.

Key Results

The HAA model achieved state-of-the-art (SOTA) results across the board:

  • On Black-reID: The model achieved 83.8% mAP and 91.0% Rank-1 accuracy, outperforming the previous SOTA (MGN) by 4.7% and 4.3% respectively.
  • On Market1501: It reached 89.5% mAP, a 2.6% improvement over MGN.
  • On DukeMTMC-reID: It reached 80.4% mAP, a 2.0% improvement over MGN.
  • Qualitative Success: Visualizations show that while standard models get "confused" by multiple people in black jackets, HAA successfully distinguishes them by focusing on hair length and shoulder width.

Limitations and Future Work

  • Occlusion Sensitivity: While the model handles clothing loss well, it relies heavily on the head-shoulder region. If a subject is wearing a large hat, carrying an umbrella, or viewed from a high "bird's eye" angle where the head is obscured, the model's primary advantage might be neutralized.
  • Dataset Diversity: The "White Group" in the Black-reID dataset is a good start, but future work could expand this to other "uniform" scenarios, such as medical scrubs or delivery person uniforms, to see if the adaptive weighting generalizes to all low-variance clothing.

Summary updated

Abstract

Person re-identification (Re-ID) aims at retrieving an input person image from a set of images captured by multiple cameras. Although recent Re-ID methods have made great success, most of them extract features in terms of the attributes of clothing (e.g., color, texture). However, it is common for people to wear black clothes or be captured by surveillance systems in low light illumination, in which cases the attributes of the clothing are severely missing. We call this problem the Black Re-ID problem. To solve this problem, rather than relying on the clothing information, we propose to exploit head-shoulder features to assist person Re-ID. The head-shoulder adaptive attention network (HAA) is proposed to learn the head-shoulder feature and an innovative ensemble method is designed to enhance the generalization of our model. Given the input person image, the ensemble method would focus on the head-shoulder feature by assigning a larger weight if the individual insides the image is in black clothing. Due to the lack of a suitable benchmark dataset for studying the Black Re-ID problem, we also contribute the first Black-reID dataset, which contains 1274 identities in training set. Extensive evaluations on the Black-reID, Market1501 and DukeMTMC-reID datasets show that our model achieves the best result compared with the state-of-the-art Re-ID methods on both Black and conventional Re-ID problems. Furthermore, our method is also proved to be effective in dealing with person Re-ID in similar clothing. Our code and dataset are avaliable on https://github.com/xbq1994/.
BibTeX
@inproceedings{xu2020black,
  author = {Boqiang Xu and Lingxiao He and Xingyu Liao and Wu Liu and Zhenan Sun and Tao Mei},
  title = {{Black Re-ID: A Head-shoulder Descriptor for the Challenging Problem of Person Re-Identification}},
  booktitle = {ACM Multimedia},
  year = {2020},
  doi = {10.1145/3394171.3414056},
}

Loading related content...

Figure