Abstract
Video-based person re-identification (ReID) faces significant challenges from occlusions and background clutter. This paper introduces the Pyramid Spatial-Temporal Aggregation (PSTA) framework, which progressively fuses frame-level features through a hierarchical structure. By integrating spatial and temporal reference attention, PSTA effectively suppresses noise and captures both short-term and long-term dependencies across video tracklets, achieving state-of-the-art performance on major benchmarks.
Introduction
Person Re-Identification (ReID) involves matching individuals across non-overlapping camera views. While image-based methods are mature, video-based ReID provides richer temporal cues necessary for handling partial occlusions or difficult viewing angles.
Existing temporal aggregation methods often face a "Goldilocks" problem. Local methods, focusing only on adjacent frames, fail during long-range occlusions due to a lack of global context. Conversely, global methods can be "poisoned" if low-quality or occluded frames dominate the sequence. The PSTA framework addresses this by building a representation that maintains local detail while establishing robust long-term dependencies.
Methodology
The PSTA framework processes video tracklets through a multi-stage hierarchical structure designed to filter out "interference features" (e.g., background objects) while preserving "clean" features of the target person.
Pyramid Structure and STAM
Instead of aggregating all frames simultaneously, PSTA groups adjacent frames into pairs and fuses them hierarchically across three stages. The core building block is the Spatial-Temporal Aggregation Module (STAM), which utilizes two novel attention mechanisms:
- Spatial Reference Attention (SRA): Correlates features within a single frame to distinguish the person from the background.
- Temporal Reference Attention (TRA): Uses a graph-based approach to establish bidirectional similarities between adjacent frames, ensuring temporal consistency and helping the model "fill in" information lost to occlusion.
To maintain computational efficiency, the framework employs factorized attention, separating spatial and channel masks to reduce the parameter count compared to dense 3D attention models.
Experimental Results
The authors evaluated PSTA on four major benchmarks: MARS, DukeMTMC-VID, iLIDS-VID, and PRID-2011.
Performance and Robustness
PSTA achieved state-of-the-art results, notably reaching 91.5% Rank-1 accuracy on the MARS dataset and 98.3% on DukeMTMC-VID. Ablation studies confirmed that the pyramid structure and STAM module significantly contribute to these gains. Visual analysis via Grad-CAM heatmaps demonstrates that PSTA is far more robust to occlusions than baseline models, consistently focusing on the target person even when they are partially obscured.
Conclusion
The PSTA framework provides a robust solution for video-based ReID by leveraging a hierarchical filtering process. While it introduces some computational overhead (approx. 9M parameters), its ability to suppress interference and capture long-range dependencies sets a new standard for the field. Future work may explore more efficient attention mechanisms or transformer-based architectures to further optimize the pyramid structure.
Summary updated