Abstract
Person search suffers from the conflicting objectives of commonness and uniqueness between the person detection and re-identification tasks that make the end-to-end training of person search networks difficult. In this paper, we propose a trident network for person search that performs detection, re-identification, and part classification together. We also devise a novel end-to-end training method using adaptive gradient weighting that controls the flow of backpropagated gradients through the re-identification and part classification networks according to the quality of the person detection. The proposed method not only prevents the over-fitting but encourages to exploit fine-grained features by incorporating the part classification branch into the person search framework. Experimental results on the CUHK-SYSU and PRW datasets demonstrate that the proposed method achieves the best performance among the state-of-the-art end-to-end person search methods.
Introduction
Person search is a complex computer vision task that involves locating individuals within uncropped scene images and matching them to a specific query identity. This requires the integration of two distinct sub-tasks: Detection, which identifies all human figures in a scene, and Re-identification (Re-ID), which distinguishes between individuals.
A significant challenge in end-to-end person search is the "conflict of objectives" between these tasks. Detection relies on commonness—identifying features shared by all humans to separate them from the background. Conversely, Re-ID relies on uniqueness—identifying fine-grained features that differentiate one person from another. When these tasks share a backbone network, poor-quality detections (e.g., partial crops or background noise) can introduce "noisy" gradients into the Re-ID branch, leading to unstable training and degraded performance.
Methods
To address these issues, we propose the Trident Person Search Network (TPSN). This architecture expands the standard detection and Re-ID framework by adding a third branch for part classification and introducing a novel gradient control mechanism.
The Trident Architecture
The TPSN consists of three specialized branches:
- Detection Branch: Responsible for generating person proposals and refining bounding box coordinates.
- Re-ID Branch: Extracts a global feature vector used for identity matching across the gallery.
- Part Classification Branch: Divides person proposals into horizontal partitions to force the network to learn discriminative local features (e.g., clothing details, accessories). This branch acts as a regularizer, ensuring the shared backbone maintains high-resolution, fine-grained information.
Adaptive Gradient Weighting Function (AGWF)
The core innovation of our approach is the Adaptive Gradient Weighting Function (AGWF). In standard joint training, all detections contribute equally to the Re-ID loss. AGWF scales the backpropagated gradients of the Re-ID and Part branches based on the detection confidence score ($\alpha_i$).
- High-confidence detections allow full gradient flow to refine identity features.
- Low-confidence detections (likely noise or background) have their gradients suppressed, preventing them from corrupting the Re-ID feature space.
Optimization and Loss Functions
We utilize a revised Re-ID loss that handles both labeled and unlabeled person proposals. To maintain a large set of comparison features without excessive memory usage, we employ a look-up table and a circular queue mechanism.
Experimental Results
The TPSN was evaluated on two primary benchmarks: CUHK-SYSU (18,184 images, 8,432 identities) and PRW (11,816 images, 932 identities).
Performance Comparison
Our method achieved state-of-the-art results, outperforming 18 existing methods:
- CUHK-SYSU: TPSN reached 93.3% mAP and 94.2% Top-1 accuracy, setting a new standard for end-to-end frameworks.
- PRW: The model achieved 53.3% mAP, significantly surpassing previous end-to-end models like NAE (44.0%) and OIM (21.3%).
Ablation Study
The importance of the AGWF was confirmed through ablation experiments. Removing the adaptive weighting led to a decrease of approximately 2.2% in detection AP and 1.2% in Re-ID mAP, demonstrating its critical role in stabilizing joint training.
Conclusion
The Trident Person Search Network successfully mitigates the conflict between detection and re-identification through its three-branch architecture and adaptive gradient weighting. By treating gradient flow as a quality filter, the model prioritizes learning from high-quality samples while leveraging part-level features for better discrimination. While the current part-based approach is optimized for upright pedestrians, future work could explore more flexible part-alignment strategies to handle diverse human poses.
Summary updated