Learning Bias-Free Representation for Large-Scale Person Re-Identification

Jiaming Xu, En ZhuIEEE Access 2019 · OpenReview

Abstract

Person re-identification (re-ID) aims to match the same person across disjointed cameras. In practice, misalignment is one of the key problems that limits the re-ID accuracy. There are many causes, such as detection errors from automatic detectors, human pose changes and relative movement between people and cameras. However, most current automatic re-ID solutions are still limited in mitigating such misalignments. In this paper, we propose that the misalignment scenarios in re-ID can be divided into three basic types. To further study their effects, we visualize their negative impacts in the learned embedding space and compare the changes in the re-ID accuracy. To address these specific misalignment problems, we design three subnetworks to correct the corresponding misalignments and discuss the performance gains. Moreover, by integrating all subnetworks into one unified structure, we propose a bias-free representation learning method. Unlike previous approaches that mainly focus on one specific kind of misalignment, our proposed method can eliminate the feature bias that is introduced by multiple misalignment problems. Systematic visualizations and comparisons are conducted to demonstrate that our method can efficiently correct the feature bias during the representation learning step. Evaluations on three large-scale datasets show that our proposed bias-free representation learning method can outperform the state-of-the-art methods.

Introduction: The Challenge of Misalignment

Person re-identification is the task of matching images of the same person across different, non-overlapping camera feeds. While it is essentially an image retrieval problem for surveillance systems, real-world pedestrian images are often "misaligned," which significantly limits accuracy. The authors categorize these challenges into three distinct types:

  1. Misdetection: Errors in automatic bounding box detectors lead to missing body parts or excessive background clutter.
  2. Orientation Change: Variations in appearance based on whether a person is seen from the front, back, or side.
  3. Pose Change: Drastic changes in body posture due to activities like walking, standing, or riding a bike.
The three primary types of misalignment in person re-identification: misdetection, orientation change, and pose change.

While previous research has attempted to solve these issues, most methods only tackle one problem at a time. Because these misalignments often happen simultaneously and compound one another, failing to address them together introduces a "feature bias" that confuses the network.

Methods: A Unified Bias-Free Architecture

The proposed model is built on a standard ResNet-50 backbone but introduces three novel, integrated subnetworks to handle the three types of misalignment:

  • Affine Transformation Subnetwork (AF): To fix misdetections, this module uses a Spatial Transformer Network (STN) inserted between ResNet blocks 3 and 4. It automatically learns to scale and translate the feature map to crop out excessive background or re-center the pedestrian.
  • Orientation Sensitive Subnetwork (OS): To handle orientation changes, the network branches off early to predict the person's orientation (front, back, or side). It then uses this prediction to weight three parallel convolutional branches, allowing the network to extract orientation-specific features.
  • Attention Guided Mask Subnetwork (AGM): To mitigate pose changes, the end of the network features eight parallel attention units. These generate soft masks that focus on distinct body parts.

A key innovation is the Mask Loss, which prevents attention "collapse"—where one attention head might just look at the whole body. By penalizing an attention mask if its highly activated area exceeds 20% of the bounding box, the network is forced to specialize in localized body parts (e.g., just the torso, or just the legs) without needing explicit part annotations.

Experimental Results and Visualization

The proposed bias-free representation method achieved state-of-the-art results across three major datasets: Market1501 (94.06% Rank-1), DukeMTMC-reID (85.73% Rank-1), and CUHK03 (66.43% Rank-1).

Ablation studies revealed that correcting pose changes yielded the highest performance gains compared to fixing orientation or misdetection alone. However, the full model (correcting all three misalignments) significantly outperformed models correcting just one or two. This proves that misalignments in raw data are correlated and must be addressed holistically.

Visual comparison of misalignment types (a) and t-SNE visualizations (b) showing how the proposed method progressively aligns features into tighter identity clusters compared to the raw baseline.

Systematic visualizations using t-SNE confirmed that the learned features cluster much more tightly per identity compared to a baseline ResNet-50. As shown in the visualization above, each subnetwork contributes to reducing the overlap between different identities in the embedding space.

Conclusion and Future Work

By learning a bias-free representation, this approach eliminates feature bias introduced by multiple simultaneous misalignment problems. While the model currently relies on external datasets (like RAP) for orientation pretraining and requires careful optimization on smaller datasets, it demonstrates a robust path forward for large-scale, real-world person re-identification. Future work may explore unsupervised ways to deduce pedestrian orientation to remove the reliance on external datasets.

Summary updated

Abstract

Person re-identification (re-ID) aims to match the same person across disjointed cameras. In practice, misalignment is one of the key problems that limits the re-ID accuracy. There are many causes, such as detection errors from automatic detectors, human pose changes and relative movement between people and cameras. However, most current automatic re-ID solutions are still limited in mitigating such misalignments. In this paper, we propose that the misalignment scenarios in re-ID can be divided into three basic types. To further study their effects, we visualize their negative impacts in the learned embedding space and compare the changes in the re-ID accuracy. To address these specific misalignment problems, we design three subnetworks to correct the corresponding misalignments and discuss the performance gains. Moreover, by integrating all subnetworks into one unified structure, we propose a bias-free representation learning method. Unlike previous approaches that mainly focus on one specific kind of misalignment, our proposed method can eliminate the feature bias that is introduced by multiple misalignment problems. Systematic visualizations and comparisons are conducted to demonstrate that our method can efficiently correct the feature bias during the representation learning step. Evaluations on three large-scale datasets show that our proposed bias-free representation learning method can outperform the state-of-the-art methods.
BibTeX
@inproceedings{xu2019learning,
  author = {Jiaming Xu and En Zhu},
  title = {{Learning Bias-Free Representation for Large-Scale Person Re-Identification}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=yBUSY0pc2s},
}
Figure