Video-Based Person Re-Identification by Simultaneously Learning Intra-Video and Inter-Video Distance Metrics

Xiaoke Zhu, Xiao-Yuan Jing, Fei Wu, Hui FengIJCAI 2016

Abstract

Video-based person re-identification (re-id) faces the dual challenge of significant intra-video variations and subtle inter-video differences. This paper introduces Simultaneous Intra-video and Inter-video Distance Learning (SI²DL), a framework that learns two distinct distance metrics to address these issues concurrently. By enforcing intra-video compactness and inter-video discriminability, SI²DL achieves state-of-the-art performance on major benchmarks like iLIDS-VID and PRID 2011.

Introduction

Person re-identification is the task of recognizing the same individual across different, non-overlapping camera views. While image-based re-id is well-studied, video-based re-id offers richer information but introduces significant complexity. The authors identify two primary hurdles:

  1. Intra-video variations: A single video contains numerous frames with varying poses, occlusions, and lighting conditions, creating "noise" that obscures the person's identity.
  2. Inter-video variations: Different individuals may appear similar due to clothing, while the same individual may look different across cameras due to viewpoint shifts.
Examples of intra-video variations showing changes in pose and appearance within a single sequence.

Most existing methods fail to address these variations simultaneously. If a video representation is not "compact" (i.e., has high intra-variation), the distance between two different videos becomes an unreliable metric for identity.

Methodology: The SI²DL Framework

The proposed SI²DL framework learns a pair of metrics to transform the feature space and stabilize person representations:

  • Intra-video Distance Metric ($V$): This metric acts as a congregating force, encouraging all frames within a single video to move closer to the video's mean. This makes the representation of a single video more stable and compact.
  • Inter-video Distance Metric ($W$): This serves as a discriminant force, ensuring that the distance between videos of the same person is smaller than the distance between videos of different people.
Inter-video matching challenges where the same individual must be identified across different camera views.

Optimization and Training

The model utilizes a Video Triplet Model for training, consisting of an anchor video, a positive match, and a negative "impostor" video. Because the objective function is not jointly convex, the authors employ the Alternating Direction Method of Multipliers (ADMM) to iteratively update and converge the metrics. This approach treats each video as a "set" of spatio-temporal features, shifting the problem from simple point-to-point matching to a robust set-based distance learning problem.

Experimental Evaluation

The model was evaluated on two standard public datasets:

  • iLIDS-VID: 600 sequences of 300 persons, characterized by heavy occlusions and clothing similarities.
  • PRID 2011: 400 sequences of 200 persons, featuring significant viewpoint and illumination changes.

Key Results

SI²DL significantly outperformed existing state-of-the-art methods at the time of publication:

Dataset Metric SI²DL Result Previous Best
iLIDS-VID Rank-1 Accuracy 48.7% 44.3%
Rank-5 Accuracy 81.1% 71.7%
PRID 2011 Rank-1 Accuracy 76.7% 64.1%

The algorithm proved efficient, typically reaching convergence in fewer than five iterations.

Discussion and Insights

The core insight of this research is the "Compactness" Prerequisite: inter-video matching becomes significantly easier once intra-video noise is mitigated. By forcing a video to be more compact, its mean becomes a much more reliable descriptor. Furthermore, the authors demonstrate that learning a pair of metrics is superior to a single common metric, as it allows the model to handle noise reduction and identity separation independently.

Limitations

While effective, SI²DL relies on the quality of underlying spatio-temporal features. Additionally, the construction of video triplets can become computationally expensive as dataset sizes grow, suggesting a need for more efficient triplet sampling strategies in future work.

Summary updated

BibTeX
@inproceedings{zhu2016video,
  author = {Xiaoke Zhu and Xiao-Yuan Jing and Fei Wu and Hui Feng},
  title = {{Video-Based Person Re-Identification by Simultaneously Learning Intra-Video and Inter-Video Distance Metrics}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure