Abstract
One-step Weakly Supervised Person Search (WSPS) jointly performs pedestrian detection and person Re-IDentification (ReID) only with bounding box annotations, which makes the traditional person ReID problem more suitable and efficient for real-world applications. However, this task is very challenging due to the following reasons: 1) large feature gap between person ReID and general object detection tasks when learning shared representations; 2) difficult pseudo identity estimation for each person image with unrefined raw detection and dramatic scale changes. To address above issues, we propose a multi-granularity graph convolution framework to jointly optimize the aligned task features, as well as to assist the pseudo label estimation. Specifically, the multi-granularity feature alignment module (MFA) in the designed two-branch framework, employs cluster-level bi-directional interaction of various granularity information to narrow down the large feature gap. Further, upon the MFA module, we introduce the multi-granularity graph-convolution-based pseudo-label estimation module, to enhance feature representations for distinguishing diverse identities. Extensive experimental results demonstrate the effectiveness of the proposed method, and show superior performances to state-of-the art methods by a large margin on CUHK-SYSU and PRW datasets.
Introduction
Person Search is a dual-task challenge requiring a model to first detect a person in a scene and then identify (ReID) them across different camera views. While fully supervised methods are effective, they rely on expensive, manually labeled identity data. Weakly Supervised Person Search (WSPS) aims to train models using only bounding box annotations, eliminating the need for identity labels.
However, one-step WSPS methods face two major hurdles. First, there is a significant feature gap: detection requires robustness to subtle appearance changes, while ReID demands fine-grained details. Second, the lack of ground-truth identities necessitates pseudo-labeling via clustering, which is often noisy due to unrefined detections and dramatic scale changes.
Proposed Methodology
The proposed framework addresses these challenges through a two-pronged strategy focused on feature alignment and refined label estimation.
1. Multi-granularity Feature Alignment (MFA)
The framework utilizes a two-stream network consisting of a Search branch (processing the whole scene for context) and an Instance branch (focusing on cropped person patches for detail).
- Cluster-level Multi-granularity Alignment (CMA): This module employs an "inter-InfoNCE" loss to encourage bi-directional information sharing between the two branches at the cluster level, narrowing the gap between context-heavy and detail-heavy features.
- Box-drift Invariance Learning (BIL): To increase robustness against noisy detections, the authors introduce random "drift" to bounding boxes during training, forcing the model to learn features that remain consistent despite localization errors.
2. Multi-granularity Graph-convolution-based Pseudo-label Estimation (MGPE)
To improve the accuracy of identity estimation, the authors construct a graph where each person is a node.
- Neighbor-Guided Sparse Graph Construction (NGSGC): Instead of a dense graph, they use a "consistency hypothesis" where a neighbor is only considered reliable if it is close to a person's representation in both the Search and Instance branches.
- Offline Graph Convolution (OGC): Graph convolutions propagate information between these reliable neighbors, smoothing the feature space and making subsequent clustering significantly more accurate for generating pseudo-labels.
Experimental Results
The method was evaluated on the CUHK-SYSU and PRW benchmarks. The proposed approach significantly outperformed previous state-of-the-art (SOTA) one-step WSPS methods. On the PRW dataset, it achieved 37.6% mAP, a 4.2% improvement over the previous best method (DICL). On CUHK-SYSU, it reached 88.4% mAP and 89.9% Top-1 accuracy.
As shown in the performance analysis, the model maintains a higher performance lead over competitors even as the "gallery size" (the number of people to search through) increases from 50 to 4,000. This demonstrates the framework's robustness for real-world, large-scale applications.
Conclusion
This research demonstrates that by carefully aligning multi-granularity features and refining labels through graph convolutions, one-step weakly supervised methods can approach the performance of more complex pipelines. The key insight lies in the power of consistency: requiring neighbors to be similar in both context and detail effectively filters out distractors. Future work may focus on reducing the computational complexity of similarity matrix construction for even larger datasets and further closing the gap between weakly and fully supervised performance.
Summary updated