Unsupervised Tracklet Person Re-Identification

Minxian Li, Xiatian Zhu, Shaogang Gong

TL;DR

This paper introduces Unsupervised Tracklet Association Learning (UTAL), a framework that allows person re-identification (re-id) models to learn directly from raw, unlabelled video data. By treating automatically generated tracklets as "noisy" labels and iteratively discovering cross-camera matches, the authors achieve state-of-the-art unsupervised performance without requiring any manual identity tagging.

What Problem Does This Paper Address?

Person Re-Identification (re-id) is the task of matching the same individual across different, non-overlapping camera views. Traditionally, this requires "supervised learning," where humans manually label thousands of image pairs across every camera combination.

This approach has a massive scalability bottleneck: as the number of cameras in a network grows, the number of potential pairs to label grows quadratically. While some researchers use "domain adaptation" (training on a labeled dataset and transferring to an unlabeled one), this assumes the two environments look similar. This paper tackles the "pure" unsupervised challenge: How can a model learn to recognize people across a camera network using only raw video and no human labels?

Methods

The authors propose the UTAL framework, which operates on tracklets—short sequences of a person's movement captured by a single camera's automated tracker. The framework has two main engines:

  1. Per-Camera Tracklet Discrimination (PCTD): The model treats each camera as a separate "task" in a multi-task learning setup. Initially, every tracklet is given a unique ID. However, because trackers often fail (fragmentation), the same person might be split into multiple tracklets. To fix this, UTAL uses "Soft Labelling." Instead of a rigid ID, it assigns a probability distribution based on visual similarity. If two tracklets in the same camera look very similar, the model learns to treat them as the same person.

  2. Cross-Camera Tracklet Association (CCTA): This is the "global" step. The model searches for reciprocal nearest neighbors across different cameras. If Tracklet A in Camera 1 thinks Tracklet B in Camera 2 is its best match, and vice versa, the model assumes they are the same person and pulls their feature representations closer together in the mathematical space.

The training follows a curriculum learning strategy: it starts by learning local features (PCTD) and only introduces the more difficult cross-camera associations (CCTA) once the model is "mature" enough to avoid being confused by false matches.

Datasets

The researchers evaluated UTAL on eight major benchmarks:

  • Video-based: iLIDS-VID, PRID2011, and MARS.
  • Image-based: CUHK03, Market-1501, DukeMTMC-ReID, and MSMT17.
  • New Benchmark: The authors introduced DukeTracklet (DukeMTMC-SI-Tracklet). Unlike other datasets that use "cleaned" data, this one uses raw, automatically generated tracklets containing real-world tracking errors and occlusions to test how the model handles imperfect data.

Key Insights and Ideas

  • Fragmentation as Information: Instead of seeing broken trajectories (fragmentation) as a failure, UTAL uses them. By applying "soft labels" to fragmented tracklets, the model discovers the underlying manifold of a person's appearance.
  • Implicit Cross-View Learning: Even before the model explicitly tries to match people across cameras, the multi-task setup (sharing a feature extractor across all cameras) forces the model to learn "view-invariant" features that work everywhere.
  • The "Catch-22" of Unsupervised Re-ID: You need good features to find matches, but you need matches to learn good features. UTAL solves this by starting with easy, high-confidence local matches and progressively moving to harder, cross-camera ones.

Key Results

UTAL significantly outperformed existing unsupervised and domain-adaptation methods:

  • Market-1501: Achieved 69.2% Rank-1 accuracy, beating the previous best unsupervised method (HHL) by 7%.
  • MARS: On this large-scale video dataset, it reached 49.9% Rank-1, a 3.1% improvement over the previous state-of-the-art.
  • Robustness: The model proved highly effective on the new DukeTracklet dataset, showing it can handle the "messy" data produced by real-world automated trackers.

Limitations and Future Work

  • The Supervised Gap: While UTAL is the best in its class, there is still a significant performance gap compared to fully supervised models (which can reach 90%+ accuracy on some datasets).
  • Initial Tracking Quality: The method relies on "off-the-shelf" detectors and trackers. If the initial tracking is extremely poor (e.g., in very crowded scenes), the "soft labels" might become too noisy for the model to converge.
  • Future Directions: The authors suggest that incorporating more complex spatio-temporal constraints (e.g., "it takes 10 seconds to walk from Camera A to Camera B") could further refine the unsupervised association process.

Summary updated

Abstract

Most existing person re-identification (re-id) methods rely on supervised model learning on per-camera-pair manually labelled pairwise training data. This leads to poor scalability in a practical re-id deployment, due to the lack of exhaustive identity labelling of positive and negative image pairs for every camera-pair. In this work, we present an unsupervised re-id deep learning approach. It is capable of incrementally discovering and exploiting the underlying re-id discriminative information from automatically generated person tracklet data end-to-end. We formulate an Unsupervised Tracklet Association Learning (UTAL) framework. This is by jointly learning within-camera tracklet discrimination and cross-camera tracklet association in order to maximise the discovery of tracklet identity matching both within and across camera views. Extensive experiments demonstrate the superiority of the proposed model over the state-of-the-art unsupervised learning and domain adaptation person re-id methods on eight benchmarking datasets.
BibTeX
@article{li2020unsupervised,
  author = {Minxian Li and Xiatian Zhu and Shaogang Gong},
  title = {{Unsupervised Tracklet Person Re-Identification}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2020},
  doi = {10.1109/tpami.2019.2903058},
}

Loading related content...

Figure