Variation Generalized Feature Learning via Intra-view Variation Adaptation

Jiawei Li, Mang Ye, Andy Jinhua Ma, Pong C. Yuen

Introduction

Person re-identification (re-ID) is a critical task in surveillance, requiring the matching of individuals across non-overlapping camera views. While supervised learning has achieved high accuracy, the dependency on massive, manually labeled datasets limits its scalability. Unsupervised video-based re-ID offers a scalable alternative by leveraging tracklets—sequences of frames of the same person within a single camera view. However, existing unsupervised methods often fall into the "invariance trap," where seeking features that remain constant across frames discards discriminative details. Furthermore, the "domain gap" between intra-view variations (seen during training) and cross-view variations (encountered during testing) remains a significant hurdle.

Variation Generalized Feature Learning (VGFL)

To address these challenges, we present Variation Generalized Feature Learning (VGFL). Instead of pursuing absolute invariance, VGFL aims to learn "adaptable" features that can generalize to unseen variations.

Video Variation Dictionary Learning

The first stage involves decomposing feature vectors into three distinct components: a person-specific identity component, a variation component modeled by a learned dictionary, and residual noise. By utilizing temporal consistency within tracklets, the model ensures that the dictionary captures meaningful structural changes, such as pose transitions, rather than random image artifacts.

Example of intra-view variation modeling where specific regions (torso in green, legs in red) are analyzed for consistency and change.

Frame-Level Adaptable Feature Learning

Rather than discarding the variation component, VGFL treats different variations as distinct "domains." By applying generalized modeling techniques, including PCA for dimensionality reduction, the framework aligns features across these domains. This process creates a representation that is inherently more robust to the types of appearance changes expected when a person moves between different camera environments.

Adaptability-Discriminability (AD) Fusion

Not all frames in a video sequence are equally informative. VGFL introduces an Adaptability-Discriminability (AD) fusion mechanism to aggregate frame-level features. This method weights frames based on two criteria:

  1. Adaptability: How well the frame's features can generalize to new views.
  2. Discriminability: How unique the features are to the specific individual.

By measuring reconstruction errors within the dictionary model, the system estimates the reliability of each frame, effectively down-weighting blurry or occluded images.

The model tracks variations across frames of the same individual to learn features that adapt to pose and viewpoint changes.

Experimental Evaluation

The VGFL framework was evaluated on three benchmark datasets: iLIDS-VID, PRID-2011, and MARS. The results demonstrate that "adaptable" features significantly outperform standard pooling methods.

Dataset Rank-1 Accuracy Previous SOTA (Unsupervised)
iLIDS-VID 50.3% 44.8% (DGM)
PRID-2011 82.0% 80.9%
MARS 51.7% 48.1% (DGM+IDE)

On the large-scale MARS dataset, VGFL achieved a 32.6% mAP, outperforming the previous state-of-the-art baselines.

Conclusion

VGFL shifts the paradigm of unsupervised re-ID from seeking invariance to fostering adaptability. By explicitly modeling intra-view variations as a proxy for cross-view changes, the framework retains more discriminative information and bridges the domain gap. Future work will explore the integration of VGFL with more advanced deep learning backbones to further narrow the performance gap between unsupervised and supervised methodologies.

Summary updated

Abstract

This paper addresses the variation generalized feature learning problem in unsupervised video-based person re-identification (re-ID). With advanced tracking and detection algorithms, large-scale intra-view positive samples can be easily collected by assuming that the image frames within the tracking sequence belong to the same person. Existing methods either directly use the intra-view positives to model cross-view variations or simply minimize the intra-view variations to capture the invariant component with some discriminative information loss. In this paper, we propose a Variation Generalized Feature Learning (VGFL) method to learn adaptable feature representation with intra-view positives. The proposed method can learn a discriminative re-ID model without any manually annotated cross-view positive sample pairs. It could address the unseen testing variations with a novel variation generalized feature learning algorithm. In addition, an Adaptability-Discriminability (AD) fusion method is introduced to learn adaptable video-level features. Extensive experiments on different datasets demonstrate the effectiveness of the proposed method.
BibTeX
@inproceedings{li2019variation,
  author = {Jiawei Li and Mang Ye and Andy Jinhua Ma and Pong C. Yuen},
  title = {{Variation Generalized Feature Learning via Intra-view Variation Adaptation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/116},
}
Figure