Abstract
Person Re-identification (ReID) plays an increasingly crucial role in modern surveillance, yet existing methods struggle with misalignment and occlusions. Most current approaches rely on external tools for body part localization or complex matching strategies, which are often unreliable due to domain gaps and high computational costs. This paper proposes the Region Generation and Assessment Network (RGANet) to effectively and efficiently detect and evaluate human body regions. RGANet introduces a Region Generation Module (RGM) that utilizes pre-trained CLIP and learnable prompts to locate body parts semantically. Additionally, a Region Assessment Module (RAM) assigns confidence scores to these regions using discrimination-aware and invariance-aware indicators, mitigating the impact of occlusions. Extensive experiments across six benchmarks demonstrate RGANet's superiority in occluded, partial, and holistic ReID tasks, achieving state-of-the-art performance with high efficiency.
Introduction: The Challenge of Occlusion and Misalignment
In real-world surveillance, matching a specific individual across different camera views is complicated by environmental factors. Pedestrians are frequently partially hidden by cars, trees, or other people (occlusion) or appear in different spatial locations within the frame (misalignment). These challenges degrade ReID performance by introducing noise and missing critical identity features.
Traditional solutions generally fall into two categories:
- External Cues: Using off-the-shelf models like pose estimators to find body parts. However, these often suffer from a domain gap between their training data and surveillance footage.
- Matching Strategies: Comparing local image patches or using graph-based representations. While accurate, these are often computationally expensive and slow during inference.
The RGANet Framework
RGANet aims to bridge the gap between semantic understanding and computational efficiency through two primary components.
Region Generation Module (RGM)
Instead of relying on heavy external pose estimators, the RGM utilizes the pre-trained CLIP (Contrastive Language-Image Pre-training) model. It generates semantic prototypes from text descriptions like "head" or "upper body." To overcome the domain gap, the module uses learnable prompts, allowing the model to optimize word embeddings specifically for ReID datasets. By calculating the cosine similarity between image pixels and these textual prototypes, the model adaptively segments specific body regions.
Region Assessment Module (RAM)
To filter out noisy or occluded regions, the RAM assigns a confidence score to each part based on two indicators:
- Discrimination-Aware Indicator: Measures how well a specific region helps distinguish between different identities.
- Invariance-Aware Indicator: Uses a memory bank to compare a detected region against the dataset's "ideal" features for that body part. If a region (e.g., a leg) is occluded by a car, it will look nothing like a typical leg, and the RAM will automatically down-weight its importance.
Key Results and Efficiency
RGANet was evaluated on six widely-used benchmarks, consistently outperforming previous state-of-the-art (SOTA) methods:
- Accuracy: On the challenging Occluded-DukeMTMC dataset, it achieved 71.6% Rank-1 accuracy and 62.4% mAP, surpassing previous leaders like DPM and FED.
- Generalization: On the large-scale MSMT17 dataset, it reached 88.1% Rank-1, a 1.9% improvement over the previous SOTA, TransReID.
- Efficiency: By bypassing external tools, RGANet runs at 213.9 FPS, making it significantly faster than methods like HOReID (62.9 FPS) and more viable for real-time applications.
Conclusion and Future Work
RGANet demonstrates that leveraging the semantic power of vision-language models like CLIP can effectively solve the problems of occlusion and misalignment in ReID without the overhead of external tools. While the current model is sensitive to certain hyper-parameters and background noise, future work could extend this semantic assessment to video-based ReID, where temporal consistency could further enhance robustness.
Summary updated