Abstract
Person re-identification (Re-ID) aims to identify pedestrians across non-overlapping camera views. While deep learning has made significant strides, many methods focus on either feature extraction or metric learning in isolation. This paper introduces a multi-view deep network architecture coupled with a Joint N-Pair Loss (JNPL) to learn view-invariant representations. By simultaneously exploiting identity classification and similarity learning across multiple positive pairs, the proposed framework achieves state-of-the-art performance on major benchmarks.
Introduction
Person re-identification is a critical challenge in computer vision, often described as a "needle in a haystack" problem. The primary difficulty lies in identifying the same individual across a network of non-overlapping cameras where lighting, pose, and viewpoints vary significantly. A person viewed from the front can appear drastically different when captured from the side or back.
Existing deep learning approaches typically fall into two categories: metric learning (e.g., Triplet Loss) and identification models (classification). However, metric learning often relies on weak binary labels, while identification models may fail to enforce a structured, view-invariant latent space. This paper proposes a joint approach that bridges these two methodologies.
Methodology: Multi-Stream Architecture
The proposed framework utilizes a multi-stream CNN architecture designed to exploit the multi-view nature of Re-ID datasets. During the training phase, $m$ different views of the same person (positive pairs) and one negative example are processed through parallel CNN branches based on a ResNet50 backbone.
Feature Aggregation and Joint Loss
Instead of simple concatenation, the network averages the feature maps from the final convolutional layer to create a compact, view-invariant representation. The core innovation is the Joint N-Pair Loss (JNPL), which combines two objectives:
- Cross-Entropy Loss ($L_{xent}$): Ensures features are discriminative for specific identities through global classification.
- N-Pair Loss ($L_{npair}$): An evolution of triplet loss that pulls multiple positive views (front, back, side) together simultaneously while pushing the negative sample away.
Key Insights: "Pull-Push" Dynamics
The JNPL method is uniquely "positive-oriented." Unlike standard N-Pair-MC losses that focus on pushing away numerous negatives, this approach prioritizes the relationship between multiple positive views. This is essential for Re-ID, where the primary hurdle is recognizing that disparate viewpoints belong to the same identity.
Joint optimization is key; the classification loss provides a global identity structure, while the N-pair loss fine-tunes the local distance metric to remain robust against viewpoint changes. Furthermore, by averaging feature maps rather than flattening them, the model maintains high accuracy with reduced dimensionality, enhancing retrieval efficiency.
Experimental Results
The JNPL framework was evaluated on three large-scale benchmarks, demonstrating significant improvements over ResNet50 baselines and existing state-of-the-art (SOTA) methods:
- Market-1501: Achieved 92.97% Rank-1 accuracy and 83.01% mAP, a nearly 10% improvement over the baseline.
- DukeMTMC-reID: Reached 83.6% Rank-1 accuracy, up from a 72.04% baseline.
- CUHK03: Achieved 84.16% Rank-1 accuracy on the challenging "detected" set.
Ablation studies confirmed the synergy of the joint loss: on Market-1501, the JNPL model outperformed an N-pair-only model by 27.89% in Rank-1 accuracy.
Conclusion
The proposed multi-view network and JNPL framework effectively eliminate complex view discrepancies by learning nonlinear, view-invariant mapping functions. While the performance gain is most pronounced in datasets with many camera views (like Market-1501), the method consistently outperforms isolated classification or metric learning approaches. Future work will explore the scalability of this multi-stream aggregation in real-world environments with even higher camera density.
Summary updated