DeepChange: A Long-Term Person Re-Identification Benchmark with Clothes Change

Peng Xu, Xiatian Zhu

Abstract

DeepChange is a large-scale, real-world benchmark designed for long-term person re-identification (re-id) where individuals change their clothes over time. Spanning 12 months and featuring over 178,000 bounding boxes across 17 cameras, it challenges the industry-standard assumption of stationary appearance. We evaluate various architectures and demonstrate that multimodal fusion—incorporating edge maps and grayscale images—is essential for identifying individuals across different seasons and outfits.

Introduction

Most person re-identification (re-id) systems operate on a "short-term" assumption: a person’s appearance (clothes, hair, accessories) remains constant as they move between camera views. While effective for tracking individuals over minutes, this assumption fails in long-term surveillance where search windows span days, weeks, or months. In the real world, people change outfits daily and styles seasonally, rendering color-based cues unreliable.

DeepChange addresses this gap by providing a massive, realistic dataset to study the "clothes-change" problem. Unlike previous datasets that were small, synthesized, or collected in controlled environments, DeepChange reflects the unconstrained reality of long-term person tracking.

The DeepChange Dataset

The core contribution of this work is the DeepChange dataset. It includes 1,121 unique individuals captured in 178,407 images across 17 outdoor cameras. The data collection spanned a full year, capturing four distinct seasons and diverse weather conditions such as snow and rain.

Overview of the DeepChange dataset illustrating the challenges of long-term re-identification across various time scales and environmental conditions.

As shown in Figure 1, the dataset tracks individuals across intervals ranging from minutes to an entire year. This longitudinal approach exposes the limitations of models that rely solely on temporary visual features like clothing color.

Methodology

To establish a baseline, the authors evaluated modern computer vision architectures, including standard CNNs (ResNet, DenseNet) and Vision Transformers (ViT). A key strategy proposed is Multimodal Fusion to mitigate the unreliability of color and texture in long-term scenarios.

Two primary fusion strategies were investigated:

  1. CNN-based Late-Fusion: Separate network branches process RGB, grayscale, and edge maps, with features concatenated at the final layer.
  2. Transformer-based Early-Fusion: Patch tokens from different modalities are fused via a weighted sum before entering self-attention layers.

By utilizing grayscale images and edge maps (generated via Edge Box), the models are encouraged to focus on "clothes-invariant" features such as body shape, gait, and physical proportions rather than just the color of a shirt.

Experimental Results

The evaluation on DeepChange reveals a significant performance gap compared to short-term datasets. For instance, while a Vision Transformer (ViT-B16) achieved 49.78% Rank-1 accuracy, its mean Average Precision (mAP) was only 14.98%. This discrepancy highlights the difficulty of finding all instances of a person across different outfits, which forms a "long tail" in search results.

Statistical distribution of (a) person IDs per camera, (b) bounding boxes per camera, and (c) person IDs across multiple cameras.

Figure 4 provides a quantitative breakdown of the dataset. The results show that structural information from edge maps provides a complementary effect to RGB data, helping models remain robust when outfit colors shift entirely. Vision Transformers generally outperformed traditional CNNs, suggesting that global self-attention mechanisms are better at capturing the subtle, invariant cues needed for long-term re-id.

Conclusion

DeepChange demonstrates that long-term re-identification remains an extremely challenging and unsolved problem. Current state-of-the-art models achieve only ~15% mAP, indicating substantial room for improvement. Future work will focus on extending the dataset collection to a three-year span and moving toward video-based long-term re-id to capture gait and behavioral patterns.

Summary updated

Abstract

Long-term re-id with clothes change is a challenging problem in surveillance AI. Currently, its major bottleneck is that this field is still missing a large realistic benchmark. In this work, we contribute a large, realistic long-term person re-identification benchmark, termed DeepChange. Its unique characteristics are: (1) Realistic and rich personal appearance (e.g., clothes and hair style) and variations: Highly diverse clothes change and styles, with varying reappearing gaps in time from minutes to seasons, different weather conditions (e.g., sunny, cloudy, windy, rainy, snowy, extremely cold) and events (e.g., working, leisure, daily activities). (2) Rich camera setups: Raw videos were recorded by 17 outdoor varying-resolution cameras operating in a real-world surveillance system. (3) The currently largest number of (17) cameras, (1, 121) identities, and (178, 407) bounding boxes, over the longest time span (12 months). We benchmark the representative supervised and unsupervised re-id methods on our dataset. In addition, we investigate multimodal fusion strategies for tackling the clothes change challenge. Extensive experiments show that our fusion models outperform a wide variety of state-of-the-art models on DeepChange. Our dataset and documents are available at https://github.com/PengBoXiangShang/deepchange.
BibTeX
@inproceedings{xu2023deepchange,
  author = {Peng Xu and Xiatian Zhu},
  title = {{DeepChange: A Long-Term Person Re-Identification Benchmark with Clothes Change}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01028},
}
Figure