Abstract
Video-based person re-identification (Re-ID) is essential for real-world surveillance but faces significant challenges due to occlusion, motion blur, and spatial misalignment. This paper introduces the Non-local Video Attention Network (NVAN) and its optimized variant, the Spatially and Temporally Efficient NVAN (STE-NVAN). By embedding non-local attention layers at multiple feature levels and exploiting spatial-temporal redundancies—such as the upright geometry of pedestrians—the proposed models achieve state-of-the-art accuracy. Notably, STE-NVAN reduces computational costs by over 70% compared to standard non-local approaches, making it highly practical for real-time applications.
Introduction
Person re-identification involves matching a specific pedestrian across different, non-overlapping camera views. While image-based Re-ID has seen significant progress, video-based Re-ID offers a more realistic representation for surveillance. However, existing video Re-ID methods often rely on simple pooling or high-level attention mechanisms that fail to capture fine-grained correlations at intermediate feature levels. Non-local blocks are a powerful alternative for capturing these dependencies but are traditionally computationally prohibitive.
Methodology
Non-local Video Attention Network (NVAN)
The NVAN architecture utilizes a ResNet-50 backbone, into which five non-local attention layers are inserted at various stages. Unlike conventional approaches that apply attention only at the final feature stage, NVAN refines features throughout the network. This multi-level integration allows the model to learn complex spatial-temporal relationships early on, which is crucial for handling occlusions and misalignments.
Spatially and Temporally Efficient NVAN (STE-NVAN)
To address the high computational cost (FLOPs) associated with standard non-local layers, the authors propose two key reduction techniques:
- Spatial Reduction (Stripe Module): Leveraging the fact that pedestrians are typically upright, the model groups horizontal pixels into $S$ stripes. Attention is calculated between these stripes rather than individual pixels, making the cost independent of image resolution.
- Temporal Reduction (Hierarchical Structure): Recognizing the high redundancy between adjacent video frames, the model employs max-pooling to halve the temporal dimension at specific stages, reducing the workload for deeper layers.
Experimental Evaluation
Datasets
The models were evaluated on two large-scale benchmarks:
- MARS: Includes 1,261 identities and over 17,000 video tracks.
- DukeMTMC-VideoReID (DukeV): Contains 1,404 identities.
Results and Key Insights
NVAN achieved a state-of-the-art 90.0% Rank-1 accuracy on MARS, a 3.8% improvement over previous methods. The optimized STE-NVAN model demonstrated exceptional efficiency, reducing the computational requirement from 60.0G FLOPs (NVAN) to 16.5G FLOPs—a 72.7% reduction. Remarkably, STE-NVAN is more efficient than a standard ResNet-50 baseline (30.4G FLOPs) while providing superior accuracy (88.9% vs 87.3% on MARS).
Conclusion
The success of NVAN and STE-NVAN highlights the importance of multi-level attention and domain-specific optimizations in video processing. By exploiting the natural geometry of pedestrians and the inherent redundancy of video data, the authors demonstrate that high-fidelity recognition can be achieved with significantly reduced computational resources. Future work may explore the robustness of these mechanisms in scenarios involving non-upright poses or extreme camera angles.
Summary updated