View-Invariant and Similarity Learning for Robust Person Re-Identification

Jean-Paul Ainam, Ke Qin, Guisong Liu, Guangchun LuoIEEE Access 2019 · OpenReview

Abstract

Person re-identification (Re-ID) aims to identify pedestrians across non-overlapping camera views. While deep learning has made significant strides, many methods focus on either feature extraction or metric learning in isolation. This paper introduces a multi-view deep network architecture coupled with a Joint N-Pair Loss (JNPL) to learn view-invariant representations. By simultaneously exploiting identity classification and similarity learning across multiple positive pairs, the proposed framework achieves state-of-the-art performance on major benchmarks.

Introduction

Person re-identification is a critical challenge in computer vision, often described as a "needle in a haystack" problem. The primary difficulty lies in identifying the same individual across a network of non-overlapping cameras where lighting, pose, and viewpoints vary significantly. A person viewed from the front can appear drastically different when captured from the side or back.

Existing deep learning approaches typically fall into two categories: metric learning (e.g., Triplet Loss) and identification models (classification). However, metric learning often relies on weak binary labels, while identification models may fail to enforce a structured, view-invariant latent space. This paper proposes a joint approach that bridges these two methodologies.

Methodology: Multi-Stream Architecture

The proposed framework utilizes a multi-stream CNN architecture designed to exploit the multi-view nature of Re-ID datasets. During the training phase, $m$ different views of the same person (positive pairs) and one negative example are processed through parallel CNN branches based on a ResNet50 backbone.

The proposed multi-view deep network architecture for joint feature extraction and similarity learning.

Feature Aggregation and Joint Loss

Instead of simple concatenation, the network averages the feature maps from the final convolutional layer to create a compact, view-invariant representation. The core innovation is the Joint N-Pair Loss (JNPL), which combines two objectives:

  1. Cross-Entropy Loss ($L_{xent}$): Ensures features are discriminative for specific identities through global classification.
  2. N-Pair Loss ($L_{npair}$): An evolution of triplet loss that pulls multiple positive views (front, back, side) together simultaneously while pushing the negative sample away.

Key Insights: "Pull-Push" Dynamics

The JNPL method is uniquely "positive-oriented." Unlike standard N-Pair-MC losses that focus on pushing away numerous negatives, this approach prioritizes the relationship between multiple positive views. This is essential for Re-ID, where the primary hurdle is recognizing that disparate viewpoints belong to the same identity.

Geometric representation of the N-pair loss dynamics, illustrating the simultaneous pulling of multiple positive views and pushing of negative samples.

Joint optimization is key; the classification loss provides a global identity structure, while the N-pair loss fine-tunes the local distance metric to remain robust against viewpoint changes. Furthermore, by averaging feature maps rather than flattening them, the model maintains high accuracy with reduced dimensionality, enhancing retrieval efficiency.

Experimental Results

The JNPL framework was evaluated on three large-scale benchmarks, demonstrating significant improvements over ResNet50 baselines and existing state-of-the-art (SOTA) methods:

  • Market-1501: Achieved 92.97% Rank-1 accuracy and 83.01% mAP, a nearly 10% improvement over the baseline.
  • DukeMTMC-reID: Reached 83.6% Rank-1 accuracy, up from a 72.04% baseline.
  • CUHK03: Achieved 84.16% Rank-1 accuracy on the challenging "detected" set.

Ablation studies confirmed the synergy of the joint loss: on Market-1501, the JNPL model outperformed an N-pair-only model by 27.89% in Rank-1 accuracy.

Conclusion

The proposed multi-view network and JNPL framework effectively eliminate complex view discrepancies by learning nonlinear, view-invariant mapping functions. While the performance gain is most pronounced in datasets with many camera views (like Market-1501), the method consistently outperforms isolated classification or metric learning approaches. Future work will explore the scalability of this multi-stream aggregation in real-world environments with even higher camera density.

Summary updated

Abstract

Person re-identification aims to identify pedestrians across non-overlapping camera views. Deep learning methods have been successfully applied to solving the problem and have achieved impressive results. However, these methods rely either on feature extraction or metric learning alone ignoring the joint benefit and mutual complementary effects of the person view-specific representation. In this paper, we propose a multi-view deep network architecture coupled with n-pair loss (JNPL) to eliminate the complex view discrepancy and learn nonlinear mapping functions that are view-invariant. We show that the problem of the large variation in viewpoints of a pedestrian can be well solved using a multi-view network. We simultaneously exploit the complementary representation shared between views and propose an adaptive similarity loss function to better learn a similarity metric. In detail, we first extract view-invariant feature representation from n-pair of images using multi-stream CNN and then aggregate these features for predictions. Given n-positive pairs and a negative example, the network aggregate the feature map of the n-positive pairs and predicts the identity of the person and at the same time learns features that discriminate positive pairs against the negative sample. Extensive evaluations on three large scale datasets demonstrate the substantial advantages of our method over existing state-of-art methods.
BibTeX
@inproceedings{ainam2019view,
  author = {Jean-Paul Ainam and Ke Qin and Guisong Liu and Guangchun Luo},
  title = {{View-Invariant and Similarity Learning for Robust Person Re-Identification}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=83YPMxfp_BJ},
}
Figure