Unsupervised Long-Term Person Re-Identification with Clothes Change

Mingkun Li, Shupeng Cheng, Peng Xu, Xiatian Zhu, Chun-Guang Li, Jun Guo

TL;DR

Traditional person re-identification (Re-ID) systems fail when people change their clothes over time. This paper introduces Curriculum Person Clustering (CPC), an unsupervised learning framework that tackles long-term Re-ID without expensive identity labels by adaptively relaxing clustering criteria to safely group a single person's diverse outfits.

What Problem Does This Paper Address?

Most person re-identification (Re-ID) systems operate under a convenient but unrealistic assumption: people don't change their clothes. While this works for short-term tracking across a few security cameras, it completely breaks down in long-term scenarios spanning days, weeks, or months.

To handle clothing changes, recent works have turned to supervised learning. However, collecting and labeling identity data across long time horizons is incredibly expensive and tedious.

Unsupervised long-term Re-ID eliminates the need for labels but introduces a massive algorithmic headache. Without ground-truth identities, models rely on pseudo-labels generated by clustering algorithms. In long-term scenarios, these algorithms easily get confused: different people wearing similar clothes get grouped together, while the same person wearing different outfits gets split apart.

Fig. 1. Visualizing the inherent challenges of long-term per- son re-identification. All the images belong to a single person. It is clear that the difference in appearance between different clothing.

Methods

To overcome this, the authors propose Curriculum Person Clustering (CPC), a framework that dynamically adjusts its clustering criteria based on how confident it is about the underlying data clusters.

Instead of using a fixed clustering threshold, CPC adopts a "curriculum learning" strategy—moving from easy tasks to hard tasks. The pipeline consists of two main components:

  1. Feature Representation Learning: A standard ResNet50 backbone extracts visual features from unlabeled pedestrian images.
  2. Curriculum Person Clustering Module: Instead of blindly trusting pseudo-labels from standard DBSCAN clustering, CPC introduces a metric called the Relaxing Index (RI).

The RI measures the internal density and similarity of a cluster. If a cluster has a high RI, it means it is highly compact (likely consisting of the same person in the same outfit). This high confidence signals to the model that the cluster has "room to grow." A dynamic training scheduler then relaxes the clustering criteria ($\Psi$), allowing the model to gradually merge different clusters representing the same person in different outfits as training progresses.

Fig. 2. The flowchart of Curriculum Person Clustering (CPC).

Datasets

The authors evaluated their approach on DeepChange, currently the largest and most realistic benchmark for long-term person Re-ID.

  • Scale: 178,407 images, 1,121 unique identities, and 17 camera views.
  • Duration: Collected over a 12-month period, ensuring significant natural clothing variations.
  • Evaluation Metrics: Cumulative Matching Characteristics (CMC, reported as Rank-1 accuracy) and mean Average Precision (mAP).

Key Insights and Ideas

  • Harnessing Cluster Density as a Proxy for Confidence: The core "aha moment" is using intra-cluster similarity (the Relaxing Index) to quantify learning difficulty. Highly dense clusters represent low-hanging fruit (same clothes), which serve as the starting point of the curriculum.
  • Breaking Color Dependency: Standard unsupervised Re-ID models heavily overfit to color features. By adaptively expanding cluster boundaries based on density rather than fixed feature distances, CPC forces the network to discover latent, clothes-independent features (like body shape or structural details).

Key Results

CPC achieves state-of-the-art performance for unsupervised long-term Re-ID on the DeepChange dataset:

  • Outperforming Unsupervised SOTA: CPC achieved a Rank-1 accuracy of 45.9% and an mAP of 14.6%, significantly outperforming top unsupervised short-term methods like Self-Paced Contrastive Learning (SpCL) and Cluster Contrast (CC).
Comparison with the state-of-the-art unsupervised re-identification models on the DeepChange dataset.
  • Rivaling Supervised Baselines: Remarkably, this unsupervised method closed the gap to fully supervised models, even outperforming several supervised architectures like MobileNetv2 and DenseNet121.
Comparison with supervised method on DeepChange.
  • Ablation Verification: Removing the curriculum clustering strategy (Model #17) caused performance to drop to 41.8% Rank-1 and 12.4% mAP, proving that the dynamic relaxation strategy is vital to handling clothing changes.
Table 3. Ablation study on DeepChange.

Limitations and Future Work

While CPC represents a major leap forward for unsupervised long-term Re-ID, the absolute performance numbers (14.6% mAP) highlight just how difficult this task remains. The authors acknowledge that extreme appearance variations over long periods still pose a formidable challenge.

Although not explicitly detailed in the paper's conclusion, a natural next step for this research would be integrating multi-modal or structural cues—such as gait analysis or 3D body shape estimation—that are inherently invariant to clothing changes.

Summary updated

Abstract

We investigate unsupervised person re-identification (Re-ID) with clothes change, a new challenging problem with more practical usability and scalability to real-world deployment. Most existing re-id methods artificially assume the clothes of every single person to be stationary across space and time. This condition is mostly valid for short-term re-id scenarios since an average person would often change the clothes even within a single day. To alleviate this assumption, several recent works have introduced the clothes change facet to re-id, with a focus on supervised learning person identity discriminative representation with invariance to clothes changes. Taking a step further towards this long-term re-id direction, we further eliminate the requirement of person identity labels, as they are significantly more expensive and more tedious to annotate in comparison to short-term person re-id datasets. Compared to conventional unsupervised short-term re-id, this new problem is drastically more challenging as different people may have similar clothes whilst the same person can wear multiple suites of clothes over different locations and times with very distinct appearance. To overcome such obstacles, we introduce a novel Curriculum Person Clustering (CPC) method that can adaptively regulate the unsupervised clustering criterion according to the clustering confidence. Experiments on three long-term person re-id datasets show that our CPC outperforms SOTA unsupervised re-id methods and even closely matches the supervised re-id models.
BibTeX
@inproceedings{li2022unsupervised,
  author = {Mingkun Li and Shupeng Cheng and Peng Xu and Xiatian Zhu and Chun-Guang Li and Jun Guo},
  title = {{Unsupervised Long-Term Person Re-Identification with Clothes Change}},
  booktitle = {IC-NIDC},
  year = {2022},
  doi = {10.1109/ic-nidc59918.2023.10390791},
}
Figure