Abstract
Unsupervised video-based person re-identification (re-ID) methods extract richer features from video tracklets than image-based ones. The state-of-the-art methods utilize clustering to obtain pseudo-labels and train the models iteratively. However, they underestimate the influence of two kinds of frames in the tracklet: 1) noise frames caused by detection errors or heavy occlusions exist in the tracklet, which may be allocated with unreliable labels during clustering; 2) the tracklet also contains hard frames caused by pose changes or partial occlusions, which are difficult to distinguish but informative. This paper proposes a Noise and Hard frame Aware Clustering (NHAC) method. NHAC consists of a graph trimming module and a node re-sampling module. The graph trimming module obtains stable graphs by removing noise frame nodes to improve the clustering accuracy. The node re-sampling module enhances the training of hard frame nodes to learn rich tracklet information. Experiments conducted on two video-based datasets demonstrate the effectiveness of the proposed NHAC under the unsupervised re-ID setting.
Introduction and Problem Statement
Person Re-identification (re-ID) is the critical task of matching the same individual across non-overlapping camera views. While video-based re-ID provides more temporal data than static images, high-performing systems traditionally rely on massive manually labeled datasets, which are costly and difficult to scale.
Unsupervised methods attempt to solve this by using clustering to generate "pseudo-labels" for training. However, standard clustering approaches often fail to account for the internal composition of video tracklets. The authors identify two major flaws in current methods:
- Noise Frames: Detection errors or heavy occlusions create "garbage" frames. When these are averaged into a tracklet's feature representation, they bias the data, leading to incorrect merges of different identities during clustering.
- Hard Frames: Frames featuring complex poses or partial occlusions are difficult to recognize but contain high information density. Current models often fail to learn effectively from these scattered, difficult samples.
Proposed Method: NHAC
The researchers propose the NHAC (Noise and Hard frame Aware Clustering) framework. NHAC builds upon Bottom-Up Clustering (BUC) by introducing two specialized modules designed to handle the nuances of video data.
Graph Trimming Module (GTM)
Instead of treating a tracklet as a simple average of frames, GTM models it as a graph. It calculates the cosine similarity between each frame and the tracklet's center. Using a dynamic threshold based on the tracklet's internal variance, GTM identifies and prunes noise frames. This ensures that the resulting tracklet feature is "clean" before the clustering process begins, preventing "cascading errors" where noise causes distinct clusters to merge prematurely.
Node Re-sampling Module (NRM)
To improve model robustness, the NRM identifies "hard" frames—those mathematically distant from the tracklet's average. These hard frames are oversampled during training, forcing the Convolutional Neural Network (CNN) to learn more discriminative features from complex variations like side views or bent poses. This approach recognizes that "easy" samples eventually stop providing useful gradients, whereas hard samples continue to refine the model's understanding of identity.
Tracklet Triplet Loss
NHAC implements a specific loss function to ensure frames within the same tracklet are pulled together in the feature space while being pushed away from frames in different pseudo-labeled clusters.
Experimental Results
The study evaluates NHAC on two large-scale benchmarks: DukeMTMC-VideoReID and MARS. NHAC outperformed existing unsupervised state-of-the-art (SOTA) methods by significant margins:
- DukeMTMC-VideoReID: Achieved 82.8% Rank-1 accuracy and 76.0% mAP (mean Average Precision), representing an 8.6-point improvement in mAP over previous SOTA methods.
- MARS: Achieved 61.8% Rank-1 and 40.1% mAP, a 10.7-point jump in mAP compared to the baseline BUC method.
- Ablation Studies: Results proved that GTM and NRM are complementary. On DukeMTMC, GTM alone improved Rank-1 by 2%, but the combination with NRM pushed that improvement to over 8%.
Conclusion and Future Work
NHAC demonstrates that being "noise-aware" and "hard-frame-aware" is essential for unsupervised video re-ID. While the method sets new benchmarks, the authors noted that performance tends to plateau after approximately 14 clustering iterations, suggesting a risk of "cluster collapse." Future work may involve incorporating camera metadata to provide additional constraints in multi-camera environments and further refining the sensitivity of the noise relaxation parameter $\delta$.
Summary updated