Person Search by Separated Modeling and A Mask-Guided Two-Stream CNN Model

Di Chen, Shanshan Zhang, Wanli Ouyang, Jian Yang, Ying Tai

Abstract

In this work, we tackle the problem of person search, which is a challenging task consisted of pedestrian detection and person re-identification (re-ID). Instead of sharing representations in a single joint model, we find that separating detector and re-ID feature extraction yields better performance. In order to extract more representative features for each identity, we segment out the foreground person from the original image patch. We propose a simple yet effective re-ID method, which models foreground person and original image patches individually, and obtains enriched representations from two separate CNN streams. We also propose a Confidence Weighted Stream Attention method which further re-adjusts the relative importance of the two streams by incorporating the detection confidence. Furthermore, we simplify the whole pipeline by incorporating semantic segmentation into the re-ID network, which is trained by bounding boxes as weakly-annotated masks and identification labels simultaneously. From the experiments on two standard person search benchmarks i.e. CUHK-SYSU and PRW, we achieve mAP of 83.3% and 32.8% respectively, surpassing the state of the art by a large margin. The extensive ablation study and model inspection further justifies our motivation.
BibTeX
@article{chen2020person,
  author = {Di Chen and Shanshan Zhang and Wanli Ouyang and Jian Yang and Ying Tai},
  title = {{Person Search by Separated Modeling and A Mask-Guided Two-Stream CNN Model}},
  journal = {IEEE Trans. Image Process.},
  year = {2020},
  doi = {10.1109/tip.2020.2973513},
}
Figure