Dual Cluster Contrastive learning for Object Re-Identification

Hantao Yao, Changsheng Xu

TL;DR

This paper introduces Dual Cluster Contrastive (DCC), a novel framework for object re-identification (ReID) that combines two memory banks: one for individual features and another for cluster centroids. This dual approach enhances robustness against noisy data and improves the discriminative power of learned features, leading to state-of-the-art performance in both supervised and unsupervised ReID tasks.

What Problem Does This Paper Address?

Object Re-identification (ReID) is crucial for tasks like video surveillance, aiming to find a specific object across different camera views. While contrastive learning has shown promise, existing "cluster contrastive" methods often rely on updating a cluster memory with individual features. This can lead to instability, especially when encountering outlier samples or noisy labels, as a single erroneous feature can disproportionately influence the cluster representation. The challenge is to create a more stable and discriminative representation that is less susceptible to such noise.

Methods

The core innovation of DCC is its Dual Cluster Contrastive framework. It maintains two distinct memory banks:

  • Individual Cluster Memory: This bank is updated using individual features, similar to traditional cluster contrastive methods. This captures fine-grained details.
  • Centroid Cluster Memory: This bank is updated using the mean feature of each cluster. This acts as a more robust representation, averaging out the impact of individual outliers.

These two memory banks are updated using separate "backbone" networks (individual and centroid backbones) to extract features. The framework then employs two types of contrastive losses:

  1. Vanilla Contrastive Loss: Applied independently to each memory bank to learn from individual and centroid perspectives.
  2. Cross-View Consistency Constraint: This novel loss encourages knowledge exchange between the individual and centroid memories. It helps the individual backbone learn from the robust centroid information and vice-versa, leading to more discriminative embeddings.

The DCC framework can be seamlessly applied to both supervised ReID (using ground-truth labels) and unsupervised ReID (using pseudo-labels generated by clustering).

Datasets

The proposed DCC framework was evaluated on three widely-used datasets:

  • Market-1501: A popular person ReID dataset.
  • MSMT17: Another large-scale person ReID dataset.
  • VeRi-776: A vehicle ReID dataset.

The paper uses these datasets for both supervised and unsupervised ReID experiments.

Key Insights and Ideas

  • Centroid-based memory is more robust: Using the mean feature of a cluster to update its memory bank significantly reduces the impact of outlier samples compared to using individual features directly. This is akin to the difference between Stochastic Gradient Descent (SGD) and Batch Gradient Descent.
  • Complementary strengths: The individual-based memory captures fine-grained details, while the centroid-based memory provides stability. Combining them offers the best of both worlds.
  • Cross-view consistency is crucial: The cross-view consistency loss is key to enabling effective knowledge transfer between the two memory banks, enhancing the discriminative power of both individual and centroid representations.
  • Versatility: DCC is easily adaptable to both supervised and unsupervised ReID settings, making it a broadly applicable framework.

Key Results

The proposed DCC framework achieved state-of-the-art results across the evaluated benchmarks in both supervised and unsupervised settings.

  • Supervised ReID: On Market-1501, DCC achieved an mAP of 89.9% (with ResNet50) and 90.6% (with ResNet50-ibn). Similar strong performance was observed on MSMT17 and VeRi-776.
  • Unsupervised ReID: DCC also demonstrated superior performance, achieving an mAP of 83.4% (ResNet50) and 85.8% (ResNet50-ibn) on Market-1501.
  • Robustness: DCC showed insensitivity to batch size variations and faster training convergence compared to existing methods.

Limitations and Future Work

The authors acknowledge that maintaining two separate memory banks might present limitations for extremely large-scale datasets. As a future direction, they aim to explore methods for achieving the benefits of both memory banks using a single memory structure, potentially through adaptive optimization strategies.

Summary updated

Abstract

Recently, cluster contrastive learning has been proven effective for object ReID by computing the contrastive loss between the individual features and the cluster memory. However, existing methods that use the individual features to momentum update the cluster memory will fluctuate over the training examples, especially for the outlier samples. Unlike the individual-based updating mechanism, the centroid-based updating mechanism that applies the mean feature of each cluster to update the cluster memory can reduce the impact of individual samples. Therefore, we formulate the individual-based updating and centroid-based updating mechanisms in a unified cluster contrastive framework, named Dual Cluster Contrastive framework (DCC), which maintains two types of memory banks: individual and centroid cluster memory banks. Significantly, the individual cluster memory considers just one individual at a time to take a single step for updating. The centroid cluster memory applies the mean feature of each cluster to update the corresponding cluster memory. During optimization, besides the vallina contrastive loss of each memory, a cross-view consistency constraint is applied to exchange the benefits of two memories for generating a discriminative description for the object ReID. Note that DCC can be easily applied for unsupervised or supervised object ReID by using ground-truth labels or the generated pseudo-labels. Extensive experiments on three benchmarks, \emph{e.g.,} Market-1501, MSMT17, and VeRi-776, under \textbf{supervised Object ReID} and \textbf{unsupervised Object ReID} demonstrate the superiority of the proposed DCC.
BibTeX
@misc{yao2021dual,
  author = {Hantao Yao and Changsheng Xu},
  title = {{Dual Cluster Contrastive learning for Object Re-Identification}},
  year = {2021},
  eprint = {2112.04662},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2112.04662},
}
Figure