Multi-Attribute Enhancement Network for Person Search

Lequan Chen, Wei Xie, Zhigang Tu, Yaping Tao, Xinming Wang

Abstract

Person search involves the dual challenge of pedestrian detection and person re-identification (Re-ID). Traditional methods often rely on global features, which are susceptible to background noise and spatial misalignment. This paper introduces the Multi-Attribute Enhancement (MAE) Network, an end-to-end framework that incorporates semantic attribute learning—such as head, clothing, and bags—to guide the model toward more discriminative local features. By leveraging a Global Attention Block (GABlock) and a Local Fusion Block (LFBlock), the MAE network achieves state-of-the-art results, notably reaching 91.8% mAP on the CUHK-SYSU benchmark.

Introduction

The task of person search requires identifying a target individual within a large gallery of raw images, necessitating both accurate detection and robust matching. Current models typically extract a single global feature vector for each detected bounding box. However, these global representations often capture irrelevant background clutter or fail when the detector's bounding box is slightly misaligned with the person's actual position.

While the Re-ID field has successfully utilized semantic attributes (e.g., "red shirt," "carrying a backpack") to improve matching accuracy, the person search community has largely avoided this due to the computational difficulty of generating and aligning these labels in uncropped scenes. The MAE network addresses this gap by introducing a systematic way to integrate attribute-level information into the search process.

Methodology

The MAE network is built upon the Faster R-CNN detector and the Norm-Aware Embedding (NAE) framework. Its architecture is enhanced by two primary modules designed to refine feature representation.

Global Attention and Local Fusion

The Global Attention Block (GABlock) utilizes a global mask to suppress background noise, ensuring the network focuses on pixels belonging to the human subject. This reduces the impact of redundant environmental context.

The Local Fusion Block (LFBlock) represents the core innovation of the MAE network. It extracts local features based on five semantic attribute categories: head, upper-body clothing, lower-body clothing, shoes, and bags. These local features are fused with the global representation to create a robust, multi-dimensional embedding. This approach treats attributes as "alignment anchors," ensuring that comparisons between individuals are spatially consistent even across different poses or viewpoints.

Comparison between standard ReID and the proposed Multi-Attribute Enhancement (MAE) framework for person search.

Attribute Label Generation

To facilitate training, the authors generated semantic attribute masks for the CUHK-SYSU and PRW datasets using a pre-trained HRNet human parsing model. This automated process provides the necessary ground truth for the LFBlock without requiring manual annotation of the entire dataset. Notably, the authors have released this whole-scene multi-attribute label set to the research community.

Experimental Results

The MAE model was evaluated on the CUHK-SYSU and PRW benchmarks, demonstrating superior performance compared to existing end-to-end methods.

  • CUHK-SYSU: The model achieved 91.8% mAP and 93.0% Rank-1 accuracy, significantly outperforming the NAE baseline.
  • PRW: The model reached 40.1% mAP and 79.7% Rank-1 accuracy.

A key finding was the model's stability as the gallery size increased. While competing models often show a sharp decline in accuracy when searching through thousands of images, the MAE network maintained a more consistent performance profile. Qualitative analysis further indicates that the model is particularly effective at matching individuals across significant viewpoint changes, such as matching a front-view query to a back-view gallery image.

Conclusion

The Multi-Attribute Enhancement Network demonstrates that integrating semantic attributes into the person search pipeline effectively mitigates the issues of background noise and misalignment. By focusing on specific body parts and accessories, the model creates more reliable embeddings for re-identification. Future work aims to develop a more unified framework that learns these local representations directly, eliminating the need for pre-generated masks and further streamlining the search process.

Summary updated

BibTeX
@inproceedings{chen2021multi,
  author = {Lequan Chen and Wei Xie and Zhigang Tu and Yaping Tao and Xinming Wang},
  title = {{Multi-Attribute Enhancement Network for Person Search}},
  booktitle = {IJCNN},
  year = {2021},
  doi = {10.1109/ijcnn52387.2021.9533620},
}
Figure