Abstract
Person search has drawn increasing attention due to its real-world applications and research significance. It aims to find a probe person in a gallery of scene images, with applications ranging from criminal investigations to missing person searches. This survey analyzes highly influential methods across three significant challenges: discriminative person features, the query-person gap, and detection-identification inconsistency. We summarize evaluation results and discuss promising future research directions.
Introduction: The Person Search Problem
In surveillance and forensics, the task of finding a specific individual is rarely as simple as matching two perfectly cropped photos. While standard Person Re-identification (Re-ID) assumes that detection is already completed and focuses on matching pre-cropped images, Person Search is an "end-to-end" task. It requires models to simultaneously detect people within raw scene images and identify the specific target.
This task is significantly harder due to several factors:
- Detection Errors: Poorly cropped bounding boxes (e.g., missing limbs or excessive background) can degrade identification performance.
- Conflicting Objectives: Detection aims for commonality (finding "any person"), whereas identification requires distinctiveness (finding "this specific person").
- The Modality Gap: In text-based search, models must bridge the semantic gap between natural language descriptions and raw pixel data.
Methodological Pillars
The survey categorizes existing literature into three strategic pillars designed to address these complexities. These are broadly divided into image-based and text-based query modalities.
1. Deep Feature Representation Learning
These methods focus on extracting "clutter-proof" features. While early approaches relied on global features, modern techniques utilize local/part-based features—dividing the body into regions—or context cues from the surrounding scene to enhance discriminability against complex backgrounds.
2. Deep Metric Learning
Metric learning addresses the "query-person gap." A landmark development in this area is the Online Instance Matching (OIM) loss. Unlike standard Softmax, which struggles with the vast number of identities in person search datasets, OIM maintains a lookup table of features to efficiently compare queries against both labeled and unlabeled identities.
3. Identity-driven Detection
This novel category integrates identity information directly into the detection process. Rather than treating detection and identification as isolated steps, these models use identity cues to "guide" the detector. For instance, Knowledge Distillation (KD) can be used where a pre-trained identification model teaches the detection network which regions of a bounding box are most critical for recognition.
Key Challenges and Insights
The survey highlights the Inconsistency Paradox: a high "Intersection over Union" (IoU) score—the standard metric for detection—does not always correlate with better identification. A box that perfectly bounds a person might include background noise that confuses the ID model.
Furthermore, while text-based search is highly practical (as it doesn't require an initial photo), it is technically demanding. Top-performing models like ViTAA succeed by decomposing both text and images into fine-grained attributes (e.g., "short hair," "blue jeans") to achieve precise cross-modal alignment.
Benchmarks and State-of-the-Art
The field is evaluated across several primary benchmarks:
- CUHK-SYSU & PRW: The standard large-scale benchmarks for image-based search.
- LSPM (Large-Scale Person Matching): Designed to test performance on partial body detections.
- CUHK-PEDES: The gold standard for text-based search, featuring over 80,000 descriptions.
Currently, image-based methods like TCTS and DKD lead with mAP scores over 93% on CUHK-SYSU. In the text-based domain, ViTAA holds the top spot, demonstrating the superiority of attribute-level alignment over global matching.
Future Frontiers
The authors identify several open research directions:
- Multi-modal Search: Combining image and text queries (e.g., "Find this person, but wearing a different hat").
- Zero-Shot Search: Improving the ability to find identities not seen during training.
- Efficiency: Optimizing models for real-time deployment across large-scale camera networks without sacrificing precision.
Summary updated