Abstract
Person search involves localizing a target individual within a diverse gallery of images. A significant challenge in this task is "scene noise"—background and foreground elements that corrupt the person's identity features as they move through different environments. We present Scene-Adaptive Person Search (SEAS), a model that utilizes bilateral modulations to eliminate scene noise and maintain consistent person representations. SEAS introduces a Background Modulation Network (BMN) to reduce background noise within detected bounding boxes and a Foreground Modulation Network (FMN) to mitigate environmental effects like lighting. Our approach achieves state-of-the-art performance on the CUHK-SYSU (97.1% mAP) and PRW (60.5% mAP) benchmarks.
Introduction: The Challenge of Scene Noise
In person search, the goal is to detect a person and then correctly identify them (re-identification) across different camera views. However, a person's captured appearance is rarely "pure." When a bounding box is generated, it often includes background noise (e.g., street elements, vehicles) and is affected by foreground noise (e.g., shadows, lighting changes).
Existing methods often either ignore these environmental factors or "bind" the scene to the person's identity. The latter is problematic because if the person moves to a new location, the previously learned scene information becomes misleading. SEAS shifts this paradigm by using scene information as a filter to clean the person's features rather than as an additive component.
Proposed Method: Scene-Adaptive Person Search (SEAS)
The SEAS framework employs a bilateral modulation strategy to isolate and remove noise at two levels:
1. Background Modulation Network (BMN)
The BMN focuses on noise within the detected bounding box. It utilizes a Multi-Granularity Embedding (MGE) to partition the person's image into horizontal strips, preserving fine-grained local details. To suppress non-person pixels, it implements a Background Noise Reduction (BNR) loss. This loss is "norm-aware," using the L2-norm of features to distinguish between high-information identity features and low-information background clutter without requiring expensive pixel-level segmentation.
2. Foreground Modulation Network (FMN)
The FMN addresses broader environmental noise. By extracting a feature map of the entire scene and applying a cross-attention mechanism, the FMN calculates a "clutter reduction offset." This offset is added to the person's embedding to effectively "cancel out" the impact of varying lighting or weather conditions, ensuring the identity representation remains stable across different scenes.
3. Bidirectional Online Instance Matching (BOIM) Loss
To enhance the re-identification sub-task, SEAS uses the BOIM loss, which combines standard Online Instance Matching (OIM) with a triplet loss. This dual approach helps the model better distinguish between similar-looking individuals (inter-person differences) while recognizing the same person across different poses and viewpoints (intra-person similarity).
Experimental Results
SEAS was evaluated on two primary benchmarks: CUHK-SYSU and PRW (Person Search in the Wild).
| Dataset | mAP | Top-1 Accuracy |
|---|---|---|
| CUHK-SYSU | 97.1% | 97.8% |
| PRW | 60.5% | - |
These results represent new state-of-the-art (SOTA) benchmarks. Notably, SEAS demonstrated superior robustness in cross-camera scenarios, confirming that its "noise cancellation" strategy effectively decouples identity from environmental context.
Conclusion
By treating scene information as a source of noise to be modulated out rather than features to be integrated, SEAS provides a more robust and adaptive solution for person search. While future work may address computational overhead and extreme occlusions, the bilateral modulation approach sets a new standard for maintaining consistent identity representations in dynamic environments.
The code is available at: https://github.com/whbdmu/SEAS.
Summary updated