Person Re-identification by Contour Sketch under Moderate Clothing Change

Qize Yang, Ancong Wu, Wei-Shi Zheng

Abstract

Person re-identification (re-id), the process of matching pedestrian images across different camera views, is an important task in visual surveillance. Substantial development of re-id has recently been observed, and the majority of existing models are largely dependent on color appearance and assume that pedestrians do not change their clothes across camera views. This limitation, however, can be an issue for re-id when tracking a person at different places and at different time if that person (e.g., a criminal suspect) changes his/her clothes, causing most existing methods to fail, since they are heavily relying on color appearance, and thus, they are inclined to match a person to another person wearing similar clothes. In this work, we call the person re-id under clothing change the "cross-clothes person re-id." In particular, we consider the case when a person only changes his clothes moderately as a first attempt at solving this problem based on visible light images; that is, we assume that a person wears clothes of a similar thickness, and thus the shape of a person would not change significantly when the weather does not change substantially within a short period of time. We perform cross-clothes person re-id based on a contour sketch of person image to take advantage of the shape of the human body instead of color information for extracting features that are robust to moderate clothing change. To select/sample more reliable and discriminative curve patterns on a body contour sketch, we introduce a learning-based spatial polar transformation (SPT) layer in the deep neural network to transform contour sketch images for extracting reliable and discriminant convolutional neural network (CNN) features in a polar coordinate space. An angle-specific extractor (ASE) is applied in the following layers to extract more fine-grained discriminant angle-specific features. By varying the sampling range of the SPT, we develop a multistream network for aggregating multi-granularity features to better identify a person. Due to the lack of a large-scale dataset for cross-clothes person re-id, we contribute a new dataset that consists of 33,698 images from 221 identities. Our experiments illustrate the challenges of cross-clothes person re-id and demonstrate the effectiveness of our proposed method.

Introduction

Standard person re-identification (re-id) systems rely heavily on color and texture, causing them to fail when a subject changes their outfit. In real-world surveillance—especially in long-term tracking or criminal investigations—this assumption is often false. A suspect might change a jacket or put on a sweater to evade detection. Because current deep learning models are "color-hungry," they often mistakenly match people based on similar clothing rather than identity. The authors define this specific challenge as "cross-clothes person re-id."

The challenge of cross-clothes re-id: (a) matching scenarios, (b) accuracy drop under clothing change, and (c) the stability of contour sketches.

Methodology

The authors argue that while clothes change, the underlying human body shape remains relatively consistent under moderate changes. To exploit this, they move away from raw RGB images and focus on contour sketches. The core of their technical contribution is a three-part architecture:

  1. Spatial Polar Transformation (SPT) Layer: Standard CNNs struggle to capture the nuances of human contours in Cartesian coordinates. The SPT layer "unrolls" the contour sketch into a polar coordinate space. Crucially, this transformation is learnable; the network learns to sample more heavily from discriminative angular regions (like the shoulders or waist) rather than sampling uniformly.
  2. Angle-Specific Extractor (ASE): Once in polar space, the feature map is divided into horizontal "angle stripes." The ASE uses a gating mechanism to model inter-channel dependencies within these stripes, effectively reweighting features to focus on the most reliable curve patterns for a specific body part.
  3. Multistream Network: To capture both global shape and fine details, the model uses parallel streams focusing on different angular ranges to extract multi-granularity features.
Visual comparison of (a-d) unstable color histograms, (e-f) RGB vs. contour sketches, and (g-h) the resulting polar-transformed feature maps.

Experimental Results

The authors contributed the PRCC (Person Re-id under moderate Clothing Change) dataset, containing 33,698 images of 221 identities. It includes views where people wear the same clothes and views where they have changed, providing a rigorous test for cross-clothing matching.

Key results include:

  • Cross-Clothes Performance: On the PRCC dataset, the proposed model achieved a 34.38% Rank-1 accuracy in the cross-clothes scenario, significantly outperforming the best RGB-based baseline (PCB) at 22.86%.
  • Ablation Success: Removing the SPT layer caused the Rank-1 accuracy to drop from 34.38% to 25.74%, proving that the polar transformation is essential for extracting shape-based identity cues.
  • Failure of RGB: Powerful models like ResNet-50 see their accuracy plummet from ~75% to ~19% on the PRCC dataset when the subject changes clothes, highlighting the necessity of shape-based approaches.

Conclusion

By leveraging body shape through contour sketches and learnable polar transformations, this work provides a robust solution for person re-identification in long-term surveillance where clothing changes are expected. While limited to "moderate" changes where clothing thickness is similar, it marks a significant step toward color-invariant biometric tracking.

Summary updated

Abstract

Person re-identification (re-id), the process of matching pedestrian images across different camera views, is an important task in visual surveillance. Substantial development of re-id has recently been observed, and the majority of existing models are largely dependent on color appearance and assume that pedestrians do not change their clothes across camera views. This limitation, however, can be an issue for re-id when tracking a person at different places and at different time if that person (e.g., a criminal suspect) changes his/her clothes, causing most existing methods to fail, since they are heavily relying on color appearance and thus they are inclined to match a person to another person wearing similar clothes. In this work, we call the person re-id under clothing change the "cross-clothes person re-id". In particular, we consider the case when a person only changes his clothes moderately as a first attempt at solving this problem based on visible light images; that is we assume that a person wears clothes of a similar thickness, and thus the shape of a person would not change significantly when the weather does not change substantially within a short period of time. We perform cross-clothes person re-id based on a contour sketch of person image to take advantage of the shape of the human body instead of color information for extracting features that are robust to moderate clothing change. Due to the lack of a large-scale dataset for cross-clothes person re-id, we contribute a new dataset that consists of 33698 images from 221 identities. Our experiments illustrate the challenges of cross-clothes person re-id and demonstrate the effectiveness of our proposed method.
BibTeX
@article{yang2019person,
  author = {Qize Yang and Ancong Wu and Wei-Shi Zheng},
  title = {{Person Re-identification by Contour Sketch under Moderate Clothing Change}},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2019},
  doi = {10.1109/tpami.2019.2960509},
}
Figure