Unified Multifaceted Feature Learning for Person Re-Identification

Cheng Yan, Guansong Pang, Xiao Bai, Chunhua ShenarXiv 2019 · OpenReview

TL;DR

This paper introduces the Unified Multifaceted Feature Learning (UMFL) framework, a novel approach that learns to identify individuals across different camera views by focusing on diverse body parts using a single, simplified neural network. UMFL achieves state-of-the-art performance by combining a unique data augmentation technique (compound batch erasing) with a hierarchical loss function, outperforming complex multi-branch models and even generalizing to vehicle re-identification.

What Problem Does This Paper Address?

Person re-identification (ReID) is the task of matching individuals across different camera feeds. A key challenge in ReID is that people can look similar, and their appearance can change significantly due to viewpoint, lighting, and clothing. To overcome this, effective ReID models need to learn "multifaceted features" – information from various parts of a person's body, like their head, torso, and legs. Traditional methods often tackle this by using complex multi-branch neural networks, where each branch specializes in a specific body part. While these methods are effective, they lead to models with a large number of parameters, making them computationally expensive and difficult to train, especially with limited labeled data. This paper aims to achieve robust multifaceted feature learning with a simpler, unified single-branch network.

Methods

The core of UMFL lies in two collaborative modules:

  • Compound Batch Erasing: This is a novel data augmentation technique that generates two types of augmented image sub-batches from the same original images:
  • Batch-constant Erasing (BcE): A specific horizontal strip of the image is erased consistently across all images within a sub-batch. This forces the network to learn from the remaining, non-erased parts of the body.
  • Random Erasing (RE): Standard random erasing is applied, where small rectangular regions are randomly removed from images. This helps the network learn globally discriminative features. These two sub-batches are then combined into a "full batch."

  • Hierarchical Structured Loss: This loss function operates on the structured batch created by compound batch erasing. It consists of:

  • Hard-triplet loss applied separately to each sub-batch (BcE and RE): This encourages learning discriminative features within each augmented view. The loss on the BcE sub-batch specifically pushes the model to focus on parts other than the erased strip, while the RE sub-batch helps retain global features.
  • Hard-triplet loss and an adapted focal loss applied to the full batch: The full batch loss helps learn finer-grained discriminative features and, crucially, addresses the challenge of distinguishing between similar-looking individuals that arise from the erasing process. The adapted focal loss specifically helps the model focus on hard-to-classify negative samples.

The UMFL framework uses a standard single-branch CNN backbone (like ResNet-50), making it significantly simpler than multi-branch architectures.

Datasets

The paper evaluates UMFL on four widely-used person ReID datasets:

  • CUHK03 (detected and labeled versions)
  • Market1501
  • DukeMTMC-ReID

Additionally, UMFL is evaluated on two vehicle ReID datasets to demonstrate its generalization capabilities:

  • VeRi-776
  • VehicleID

Key Insights and Ideas

  • Simplicity can be powerful: The paper demonstrates that a single-branch network, when equipped with the right augmentation and loss strategy, can outperform complex multi-branch architectures for learning multifaceted features.
  • Strategic erasing for focused learning: Compound batch erasing is a clever way to force the model to look at different parts of the body. Erasing a consistent strip (BcE) makes the model rely on other parts, while random erasing (RE) maintains global context.
  • Hierarchy is key for loss: Applying losses at different levels (sub-batches and full batch) allows the model to learn a richer set of features, from part-specific details to global context and cross-identity distinctions.
  • Generalization beyond person ReID: The success on vehicle ReID suggests that the core principles of UMFL (learning diverse features from different parts) are broadly applicable to other visual recognition tasks.

Key Results

  • State-of-the-art performance on person ReID: UMFL-enabled ASB (a single-branch ResNet-50 with UMFL) significantly outperforms competing methods on all four person ReID datasets, achieving improvements of up to 17.5% on CUHK03.
  • Improved existing methods: When integrated into striping-based methods like ABD and BDB, UMFL provides substantial performance gains, demonstrating its versatility.
  • Efficiency: UMFL uses a significantly simpler network structure (e.g., 25M parameters for UMFL-enabled ASB vs. 69M for ABD), leading to more efficient training.
  • Effective generalization: UMFL-enabled ASB achieves state-of-the-art results on vehicle ReID datasets, with improvements of up to 2.7% in mAP and 0.7%-0.8% in R-1 compared to the ASB baseline.
  • Qualitative improvements: Attention map visualizations show that UMFL-enabled ASB attends to more diverse and relevant body parts, ignoring background noise, whereas baseline methods focus on single, highly discriminative parts.

Limitations and Future Work

  • Limitations: The paper focuses on supervised ReID. While it shows generalization to vehicle ReID, further exploration in unsupervised or semi-supervised settings could be beneficial. The effectiveness of the specific erasing strategies might be sensitive to dataset characteristics.
  • Future Work: The authors suggest that UMFL can be plugged into complex networks to enhance their performance. Further research could explore optimal ways to integrate UMFL into various network architectures. Investigating different erasing strategies or more sophisticated hierarchical loss designs could also be fruitful. The generalizability to other domains beyond person and vehicle ReID is also an open avenue.

Summary updated

Abstract

Person re-identification (ReID) aims at re-identifying persons from different viewpoints across multiple cameras, of which it is of great importance to learn multifaceted features expressed in different parts of a person, e.g., clothes, bags, and other accessories in the main body, appearance in the head, and shoes in the foot. To learn such features, existing methods are focused on the striping-based approach that builds multi-branch neural networks to learn local features in each part of the identities, with one-branch network dedicated to one part. This results in complex models with a large number of parameters. To address this issue, this paper proposes to learn the multifaceted features in a simple unified single-branch neural network. The Unified Multifaceted Feature Learning (UMFL) framework is introduced to fulfill this goal, which consists of two key collaborative modules: compound batch image erasing (including batch constant erasing and random erasing) and hierarchical structured loss. The loss structures the augmented images resulted by the two types of image erasing in a two-level hierarchy and enforces multifaceted attention to different parts. As we show in the extensive experimental results on four benchmark person ReID datasets, despite the use of significantly simplified network structure, our method performs substantially better than state-of-the-art competing methods. Our method can also effectively generalize to vehicle ReID, achieving similar improvement on two vehicle ReID datasets.
BibTeX
@misc{yan2019unified,
  author = {Cheng Yan and Guansong Pang and Xiao Bai and Chunhua Shen},
  title = {{Unified Multifaceted Feature Learning for Person Re-Identification}},
  year = {2019},
  eprint = {1911.08651},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.08651},
}
Figure