Multi-Granularity Graph-Convolution-Based Method for Weakly Supervised Person Search

Haichun Tai, De Cheng, Jie Li, Nannan Wang, Xinbo Gao, Jin Gao

Abstract

Weakly Supervised Person Search (WSPS) aims to detect and identify individuals using only bounding box annotations, bypassing the high cost of manual identity labeling. This paper introduces a novel one-step framework that addresses the feature gap between object detection and person re-identification (Re-ID) and the noise inherent in unrefined detections. Our approach integrates Multi-granularity Feature Alignment (MFA) to synchronize scene-level and instance-level features, Box-drift Invariance Learning (BIL) to enhance robustness against localization errors, and Multi-granularity Graph-convolution-based Pseudo-label Estimation (MGPE) for accurate identity labeling. Evaluations on the CUHK-SYSU and PRW benchmarks demonstrate that our method achieves state-of-the-art results, significantly outperforming existing one-step weakly supervised methods.

Introduction

Person Search is a dual-task challenge requiring the simultaneous detection of individuals in a scene and their subsequent identification across different views. While fully supervised methods have seen success, they rely on expensive identity-labeled datasets. Weakly Supervised Person Search (WSPS) offers a scalable alternative by using only bounding box annotations. However, WSPS faces two primary challenges: the feature gap between the coarse requirements of detection and the fine-grained requirements of Re-ID, and the presence of noisy detections in one-step systems where the detector is still learning. This paper proposes a multi-granularity approach to address these hurdles effectively.

Methodology

Multi-granularity Feature Alignment (MFA)

To bridge the gap between detection and Re-ID, we employ a two-branch architecture consisting of a Search Branch for global context and an Instance Branch for fine-grained person details. We introduce Cluster-level Multi-granularity Alignment (CMA), which utilizes memory banks to store features from both branches. By applying a contrastive loss, we ensure that coarse scene features and fine instance features of the same individual are mapped closely in the feature space.

The proposed framework architecture, highlighting the feature learning and pseudo-label estimation stages.

Box-drift Invariance Learning (BIL)

One-step person search models often suffer from imprecise bounding box proposals. To mitigate this, we implement Box-drift Invariance Learning. By intentionally introducing noise (drift) to bounding boxes during training, we force the model to learn features that are invariant to slight misalignments or scale variations, enhancing its robustness during real-world inference.

Multi-granularity Graph-convolution-based Pseudo-label Estimation (MGPE)

For identity labeling, we move beyond simple clustering. We construct a sparse graph where nodes represent person features and edges denote similarity. We then apply Offline Graph Convolution (OGC) to propagate information across the graph. This process smooths the feature distributions, allowing the model to leverage neighborhood information to generate more accurate pseudo-labels for training.

Experimental Results

Datasets and Benchmarks

We evaluated our framework on two standard datasets: CUHK-SYSU, featuring over 18,000 images from diverse street and movie scenes, and PRW, which contains challenging multi-camera footage with significant viewpoint variations.

Performance Analysis

Our method significantly outperforms previous state-of-the-art one-step weakly supervised models. On the PRW dataset, we achieved a mean Averaged Precision (mAP) of 37.6%, compared to the previous best of 33.4%. On CUHK-SYSU, our model reached an mAP of 88.4% and a Top-1 accuracy of 89.9%.

Performance comparison (mAP) against varying gallery sizes on the CUHK-SYSU dataset, showing the superiority of our method.

As shown in the figure above, our model maintains superior performance even as the gallery size increases to 4,000 images, demonstrating its robustness and scalability for large-scale real-world applications.

Conclusion

This work presents a robust one-step framework for Weakly Supervised Person Search. By leveraging multi-granularity feature alignment and graph-based label estimation, we successfully address the core challenges of feature gaps and noisy detections. While a performance gap remains compared to fully supervised methods, our approach sets a new benchmark for weakly supervised systems and provides a foundation for future research into extreme scale variations and cross-camera consistency.

Summary updated

BibTeX
@inproceedings{tai2024multi,
  author = {Haichun Tai and De Cheng and Jie Li and Nannan Wang and Xinbo Gao and Jin Gao},
  title = {{Multi-Granularity Graph-Convolution-Based Method for Weakly Supervised Person Search}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=P1QLcW24To},
}
Figure