Random Occlusion Recovery for Person Re-identification

Di Wu, Kun Zhang, Si-Jia Zheng, Yongtao Hao, Fuqiang Liu, Xiao Qin, Fei Cheng, Yang Zhao, Qi Liu, Changan Yuan, De-Shuang HuangJournal of Imaging Science and Technology 2018 · arXiv

TL;DR

This paper tackles the data scarcity problem in person re-identification (ReID) by proposing a novel data augmentation technique. They use a Generative Adversarial Network (GAN) to synthesize realistic, yet not identical, de-occluded pedestrian images from occluded ones, crucially retaining their original identity labels. This allows for a significant increase in the number of training samples per identity, leading to improved ReID performance.

What Problem Does This Paper Address?

Person re-identification (ReID) is a critical task in computer vision for applications like surveillance and behavioral analysis. It involves matching a person's image across different cameras or time instances. While deep learning models have achieved impressive results, they are heavily reliant on large amounts of labeled training data. Existing ReID datasets often suffer from a limited number of images per identity, which can lead to overfitting and reduced model robustness. Manually annotating such datasets is also time-consuming and expensive. This paper addresses this data bottleneck by proposing a method to automatically generate more labeled training data.

Methods

The core of the proposed method is a Generative Adversarial Network (GAN) designed for "random occlusion recovery." The process involves two main steps:

  1. Random Occlusion: During training, input pedestrian images are randomly occluded with rectangular blocks. These blocks are filled with the mean pixel values of the dataset's R, G, and B channels to create a more naturalistic occlusion.
  2. De-occluding GAN: A GAN is trained to take these occluded images and generate de-occluded versions. The generator network, implemented as a U-Net architecture, learns to reconstruct the missing parts. The key innovation here is that the generated de-occluded images are designed to be similar but not identical to the original images. This is achieved by using a Euclidean loss function for the generator, encouraging it to produce plausible reconstructions without perfectly copying the original.
  3. Label Preservation: Crucially, the generated de-occluded images are assigned the original identity labels of the input images. This allows for a direct increase in the number of samples per identity in the training set.

The GAN consists of a generator (G) and a discriminator (D). The generator aims to produce realistic de-occluded images that can fool the discriminator, while the discriminator tries to distinguish between real and generated images.

Datasets

The proposed method was evaluated on three widely used person ReID datasets:

  • CUHK03: Contains 13,164 images of 1360 identities across six camera views.
  • Market-1501: A large-scale dataset with 32,643 annotated boxes of 1501 persons, collected from a supermarket.
  • DukeMTMC-reID: A subset of the DukeMTMC dataset, featuring 36,411 images of 1812 identities captured by eight high-resolution cameras.

The paper also mentions using a specific subset of DukeMTMC-reID for training the GAN model.

Key Insights and Ideas

  • Synthesizing "Similar but Not Identical" Images is Key: The authors hypothesize that generating images that are not exact copies of the originals is crucial. This prevents the model from simply memorizing the training data and encourages it to learn more robust features.
  • Preserving Labels is Paramount for Data Augmentation: Unlike some prior GAN-based augmentation methods that assign new labels to generated images, this work's success hinges on retaining the original identity labels. This directly addresses the problem of limited samples per identity.
  • Occlusion as a Naturalistic Augmentation Strategy: Randomly occluding images mimics real-world scenarios where pedestrians are partially obscured, making the generated data more relevant and challenging.
  • GANs as a Powerful Tool for Data Scarcity: The paper demonstrates the effectiveness of GANs not just for generating novel images, but for strategically augmenting existing datasets to improve performance on downstream tasks.

Key Results

The proposed data augmentation method, when applied to a DenseNet baseline, consistently improved ReID performance across all three datasets:

  • Market-1501 (Single Query): Achieved a Rank-1 accuracy of 91.60% and mAP of 78.86% with 12936 generated images. With re-ranking, performance further improved to 93.29% Rank-1 and 90.36% mAP. This represents a significant improvement over the baseline (89.67% Rank-1, 73.58% mAP).
  • DukeMTMC-reID: Achieved Rank-1 accuracy of 82.04% and mAP of 66.81% with the baseline. With re-ranking, performance reached 86.35% Rank-1 and 82.81% mAP.
  • CUHK03: Achieved 47.31% mAP accuracy and 50.64% Rank-1 accuracy, outperforming compared methods.

The paper also shows that adding too many GAN images can degrade performance, suggesting an optimal balance is needed.

Limitations and Future Work

  • Parameter Sensitivity: The paper acknowledges that the number of generated GAN images (M) is an important parameter. While they found an optimal point, further investigation into the sensitivity and optimal selection of M could be beneficial.
  • Future Work: The authors suggest that future work could focus on generating GAN images with varying degrees of noise or disturbance to further enhance the robustness of the baseline model. This implies exploring more complex occlusion patterns or image degradations.

Summary updated

Abstract

As a basic task of multi-camera surveillance system, person re-identification aims to re-identify a query pedestrian observed from non-overlapping multiple cameras or across different time with a single camera. Recently, deep learning-based person re-identification models have achieved great success in many benchmarks. However, these supervised models require a large amount of labeled image data, and the process of manual labeling spends much manpower and time. In this study, we introduce a method to automatically synthesize labeled person images and adopt them to increase the sample number per identity for person re-identification datasets. To be specific, we use block rectangles to randomly occlude pedestrian images. Then, a generative adversarial network (GAN) model is proposed to use paired occluded and original images to synthesize the de-occluded images that are similar but not identical to the original image. Afterward, we annotate the de-occluded images with the same labels of their corresponding raw images and use them to augment the number of samples per identity. Finally, we use the augmented datasets to train baseline model. The experimental results on CUHK03, Market-1501 and DukeMTMC-reID datasets show the effectiveness of the proposed method.
BibTeX
@article{wu2018random,
  author = {Di Wu and Kun Zhang and Si-Jia Zheng and Yongtao Hao and Fuqiang Liu and Xiao Qin and Fei Cheng and Yang Zhao and Qi Liu and Changan Yuan and De-Shuang Huang},
  title = {{Random Occlusion Recovery for Person Re-identification}},
  journal = {Journal of Imaging Science and Technology},
  year = {2018},
}
Figure