Abstract
Person search aims to locate a specific individual within unconstrained scene images, combining the tasks of person detection and person matching. Traditional Detection-Matching frameworks often struggle with "distractor" images—non-person background regions or similar-looking individuals. This paper introduces the OR Similarity metric, which integrates Objectness and Repulsion information to refine visual matching. The objectness term leverages detector confidence to penalize false detections, while the repulsion term uses the probe's spatial neighbors as "hard negatives" to prevent identity confusion. Evaluated on the CUHK-SYSU and PRW datasets, the proposed method consistently improves search accuracy across various description models, raising mAP from 92.32% to 93.23% on CUHK-SYSU and from 50.91% to 52.30% on PRW.
Introduction
The task of person search involves finding a specific individual (the "probe") within a large gallery of scene images. This is typically approached as a two-stage process: first, a detector identifies all person-like regions in the gallery images; second, a matching model compares these regions to the probe image to find the best match. However, this framework faces significant challenges from "distractors"—regions that the detector mistakenly identifies as people or individuals who look similar to the target but are not the same person.
The Challenges: Distractors and Identity Confusion
Current systems often fail because they treat every detection as a valid person and ignore the spatial context of the probe.
- Objectness Issues: Detectors often produce high-confidence boxes for background clutter or partial body parts. If these "distractor" boxes happen to have high visual similarity to the probe, they result in false positives.
- Identity Confusion (Repulsion): In crowded scenes, a probe person is often surrounded by others. These neighbors are guaranteed to have different IDs. Without accounting for this, a system might match the probe to a gallery image that actually represents one of the probe's neighbors.
Proposed Method: OR Similarity
To address these issues, the researchers propose OR Similarity, a scoring modification that can be applied to existing person search models without retraining. The final similarity score $S_{or}$ is the product of three terms:
- Visual Similarity ($S$): The standard similarity between the feature embeddings of the probe and the gallery candidate.
- Objectness Term ($S^o$): This term incorporates the confidence score from the person detector. By applying an exponential function to the detection score, the system heavily penalizes candidates that the detector is less certain are actual people, effectively filtering out background noise.
- Repulsion Term ($S^r$): This term leverages the fact that the probe person and their neighbors in the original frame must have different identities. If a gallery candidate is found to be more similar to one of the probe's neighbors than to the probe itself, the repulsion term drastically reduces its similarity score, preventing identity confusion.
Key Results
The OR Similarity metric was evaluated using six different description models on two major benchmarks. The addition of OR Similarity consistently improved performance across the board:
- CUHK-SYSU: Using the MGN model, the mean Average Precision (mAP) rose from 92.32% to 93.23%.
- PRW: On the more difficult PRW dataset, mAP increased from 50.91% to 52.30%.
Qualitative analysis demonstrated that the repulsion term is particularly effective in cases of occlusion. When a probe person is partially hidden by a neighbor, standard models often latch onto the neighbor's features; OR Similarity successfully "repels" these incorrect matches.
Conclusion
By explicitly modeling person objectness and spatial repulsion, the proposed OR Similarity metric provides a robust way to prune distractors in person search. Its "plug-and-play" nature allows it to enhance a wide variety of existing detection-matching frameworks. Future work may involve integrating these constraints directly into the representation learning phase to create even more discriminative person embeddings.
Summary updated