Abstract
Person Re-identification (Re-ID) in video sequences often suffers from appearance changes due to motion and occlusion. In the context of the ICCV 2021 VIPriors Re-identification Challenge, which prohibits pre-trained weights, data efficiency is paramount. This paper introduces Video Temporal Relationship Mining (VTRM), an iterative retrieval strategy that exploits temporal continuity in video frames. By chaining retrievals frame-by-frame rather than performing a single global search, the system maintains identity consistency across significant visual variations. Combined with a modified Multiple Granularity Network (MGN) and model ensembling, the proposed pipeline achieves a state-of-the-art 96.4% mAP on the SynergyReID benchmark.
Introduction
Standard Person Re-identification (Re-ID) systems typically operate by comparing a single query image against a vast gallery of candidate images simultaneously. While effective for static image datasets, this "one-to-many" approach ignores the inherent temporal structure of video-based data. In datasets like SynergyReID, which features basketball footage, gallery images are often continuous frames from a video sequence.
The VIPriors Re-identification Challenge adds a layer of complexity by requiring models to be trained from scratch without pre-trained weights (e.g., from ImageNet). To address these challenges, researchers from Fudan University and Meituan developed a "Video Temporal Relationship Mining" (VTRM) strategy that rethinks the retrieval process as a sequential chain rather than a parallel search.
Methodology
Architectural Enhancements: Modified MGN
The baseline architecture utilized is the Multiple Granularity Network (MGN), which extracts features by splitting images into horizontal stripes. The authors observed that the standard three-stripe split often bisected critical discriminative features, such as jersey numbers. To mitigate this, they introduced a fourth branch (Part-4), increasing the granularity of local feature extraction and ensuring that vital identity markers were preserved more effectively.
Video Temporal Relationship Mining (VTRM)
The core innovation, VTRM, leverages the fact that visual changes between adjacent video frames are minimal. Instead of using the initial query to find all matches, the system finds the single most similar frame. This retrieved frame then serves as the query for the next step. This "chaining" allows the model to navigate through occlusions or pose changes that would otherwise cause a direct comparison to fail.
The authors evaluated several VTRM variants to optimize performance and prevent error propagation:
- Local-N: The search for the next frame is guided only by the $N$ most recently retrieved images.
- Local w. ref: The original query is maintained as a constant reference alongside the local temporal window.
- Global: All previously retrieved images are used to find the next match.
Model Fusion and Training
To maximize robustness, the final system employs an ensemble of five architectures, including variants of ResNet50 and ResNet101 equipped with IBN-blocks and Squeeze-and-Excitation modules. A voting strategy determines the final retrieval order, further refining the results obtained from the VTRM process.
Experimental Results
Experiments conducted on the SynergyReID dataset demonstrate the efficacy of the proposed components. The dataset consists of basketball players and referees, where the query is the first frame of a sequence and the gallery contains all subsequent frames.
| Method | mAP (%) |
|---|---|
| Baseline MGN | 90.4 |
| MGN + Part-4 Branch | 91.8 |
| MGN + VTRM (Local-1) | 93.0 |
| Full Pipeline (Ensemble + 6x Schedule) | 96.4 |
The "Local-2" strategy was found to be the most effective, providing a balance between temporal context and resilience against "poisoned" data (incorrect retrievals that could contaminate a global search).
Conclusion
By treating video Re-ID as a temporal chaining problem, the VTRM strategy significantly outperforms standard retrieval methods in data-constrained environments. While highly effective, the method remains sensitive to the quality of the initial query; severe initial occlusions can lead to a breakdown of the retrieval chain. Future research will focus on making the iterative search more robust to these initial failures through multi-hypothesis tracking and adaptive re-anchoring.
Summary updated