TokenMatcher: Diverse Tokens Matching for Unsupervised Visible-Infrared Person Re-Identification

Xiao Wang, Lekai Liu, Bin Yang, Mang Ye, Zheng Wang, Xin XuAAAI 2025 · DOI

Abstract

Unsupervised visible-infrared person re-identification (US-VI ReID) is a challenging task due to the significant modality gap and the absence of ground-truth labels. Existing methods typically rely on a single global feature for matching, which often fails to capture the fine-grained details necessary for robust cross-modal association. We present TokenMatcher, a framework that utilizes multiple class tokens within a Vision Transformer (ViT) to achieve diverse tokens matching. By integrating information from multiple feature perspectives and employing a consistency-based neighbor learning strategy, TokenMatcher effectively mitigates noise in pseudo-labels and resolves the "split cluster" problem. Experimental results on the SYSU-MM01 and RegDB datasets demonstrate that TokenMatcher achieves state-of-the-art performance, significantly outperforming previous unsupervised approaches.

Introduction

Person Re-Identification (ReID) involves identifying the same individual across different camera views. While standard ReID focuses on visible (RGB) imagery, practical surveillance systems often utilize infrared (IR) cameras in low-light conditions. The resulting cross-modality matching problem is difficult because of the large discrepancy in visual appearance between RGB and thermal images. Unsupervised VI-ReID (US-VI ReID) attempts to learn these associations without manual annotations, typically by clustering features to generate pseudo-labels.

A major limitation of current US-VI ReID methods is their reliance on a single global feature vector for matching. This "coarse" approach often fails to distinguish between similar-looking individuals or correctly associate the same individual across modalities, leading to fragmented or "split" clusters in the embedding space.

Comparison between Single Global Matching (a) and the proposed Diverse Tokens Matching (b).

Methodology

TokenMatcher introduces a multi-token architecture to provide a more comprehensive representation of person identities through three primary components.

Diverse Tokens Matching (DTM)

Instead of a single [CLS] token, TokenMatcher employs multiple class tokens within a Vision Transformer. Each token is encouraged to learn distinct, non-redundant features through a Self-Diverse Constraint Loss. By aggregating the similarity scores across these diverse embedding spaces, the model constructs a more reliable cross-modality similarity matrix, capturing various attributes such as body shape, gait, or clothing patterns.

Diverse Tokens Neighbor Learning (DTNL)

To handle the inherent noise in unsupervised pseudo-labels, DTNL identifies "reliable neighbors" based on cross-token consistency. A sample is considered a valid neighbor only if it consistently appears in the neighborhood sets of multiple tokens. This intersection-based filtering, combined with camera-aware refinement, significantly reduces the impact of outliers and mislabeled samples.

Homogeneous Fusion (HF)

The HF module specifically targets the "split cluster" problem. When DTM identifies that multiple clusters within the same modality likely belong to the same identity, the HF module pulls these clusters together in the embedding space. This ensures that the intra-modality representation is cohesive before attempting inter-modality matching.

The overall architecture of TokenMatcher, illustrating the DTM, DTNL, and HF modules.

Experimental Results

The proposed framework was evaluated on two benchmark datasets: SYSU-MM01 and RegDB.

  • RegDB: On the Visible to Infrared task, TokenMatcher achieved a Rank-1 accuracy of 92.96%, representing a 6.05% improvement over the previous state-of-the-art method (SDCL).
  • SYSU-MM01: The model reached 65.07% Rank-1 accuracy in the All Search mode, outperforming existing unsupervised methods and rivaling some supervised baselines.

Ablation studies highlighted the importance of the DTM component. Removing DTM led to a drastic performance decline, with Rank-1 accuracy dropping by approximately 11% on SYSU-MM01 and 38% on RegDB, confirming that multi-perspective feature matching is critical for bridging the modality gap.

Conclusion

TokenMatcher demonstrates that leveraging diverse tokens within a transformer architecture significantly improves the robustness of unsupervised visible-infrared person re-identification. By moving beyond coarse global matching and utilizing consistency-based denoising, the framework effectively handles the modality gap and pseudo-label noise. Future research will focus on reducing computational overhead and further closing the performance gap with fully supervised models.

Summary updated

BibTeX
@inproceedings{wang2025tokenmatcher,
  author = {Xiao Wang and Lekai Liu and Bin Yang and Mang Ye and Zheng Wang and Xin Xu},
  title = {{TokenMatcher: Diverse Tokens Matching for Unsupervised Visible-Infrared Person Re-Identification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i8.32855},
}
Figure