Weakly Supervised Visible-Infrared Person Re-Identification via Heterogeneous Expert Collaborative Consistency Learning

Yafei Zhang, Lingqi Kong, Huafeng Li, Jie Wen

TL;DR

This paper introduces a novel weakly supervised approach for Visible-Infrared Person Re-Identification (VIReID) that eliminates the need for costly cross-modal identity labels. By training modality-specific "experts" and fostering their collaborative learning, the method effectively establishes cross-modal identity correspondences, achieving performance comparable to fully supervised methods.

What Problem Does This Paper Address?

Visible-Infrared Person Re-Identification (VIReID) is crucial for surveillance and smart city applications, aiming to match pedestrians across visible and infrared camera feeds. The primary challenge is the significant visual discrepancy between these modalities, which makes annotating cross-modal datasets extremely expensive and labor-intensive. Existing methods often rely on fully labeled cross-modal data, or struggle with noisy pseudo-labels in semi-supervised and unsupervised settings. This paper tackles the problem of VIReID under a weakly supervised setting, where only single-modality identity labels are available, thus drastically reducing annotation costs and complexity.

Methods

The core of the proposed method is a Heterogeneous Expert Collaborative Consistency Learning (HEL-CCL) framework. It comprises three main stages:

  1. Heterogeneous Expert Learning (HEL): Separate classification experts (based on ResNet-50 encoders) are trained independently for visible and infrared modalities using only their respective intra-modal identity labels. These experts learn to extract identity-discriminative features within their own modality.
  2. Cross-Modal Relationship Establishment (CRE): The trained experts are used to predict identities for samples from the other modality. By analyzing the consistency and agreement between these predictions, the framework establishes robust cross-modal identity correspondences. This is achieved by categorizing matches into consistent, unique, and conflicting, and constructing matrices (Mc, Ms, Mw) to represent these relationships.
  3. Collaborative Consistency Learning (CCL): This stage leverages the established cross-modal correspondences to guide the training of the feature encoders and a shared classifier.
  4. Cross-Modal Consistency Learning (CMCL): A cross-entropy loss is applied to enforce consistent predictions for samples with established correspondences. A relaxed loss is used for uncertain correspondences to mitigate label noise.
  5. Collaborative Learning Among Experts (CLAE): To further enhance consistency, a collaborative loss encourages experts to make similar predictions for cross-modal positive samples. Prototypes for each identity are used to facilitate this. The strength of this constraint is dynamically adjusted based on the confidence of expert predictions.

This iterative process of experts learning, establishing correspondences, and then collaboratively refining features leads to modality-invariant representations.

Datasets

The method was evaluated on two widely used VIReID datasets:

  • SYSU-MM01: A large-scale dataset with images from four visible and two near-infrared cameras, covering diverse indoor and outdoor scenes.
  • LLCM: A challenging dataset captured in low-light environments using nine visible and nine infrared cameras.

Key Insights and Ideas

  • Weak Supervision is Sufficient: The paper demonstrates that VIReID can be effectively performed with only single-modality identity labels, significantly reducing the annotation burden.
  • Expert Collaboration is Key: By treating modality-specific classifiers as "heterogeneous experts" and forcing them to collaborate and agree on cross-modal identities, robust correspondences can be learned even without direct supervision.
  • Leveraging Uncertainty: The method intelligently handles uncertain cross-modal correspondences by using a relaxed loss, preventing noisy labels from degrading performance.
  • Modality Discrepancy Mitigation: The collaborative consistency learning framework effectively bridges the gap between visible and infrared features, leading to more unified representations.

Key Results

The proposed weakly supervised method achieved performance comparable to, and in some cases surpassing, state-of-the-art fully supervised methods on both SYSU-MM01 and LLCM datasets.

  • On SYSU-MM01 (all-search), the method achieved Rank-1 accuracy of 70.4% and mAP of 66.6%, outperforming several supervised methods and significantly outperforming semi-supervised and unsupervised approaches.
  • On LLCM (VIS to IR), it achieved Rank-1 accuracy of 55.3% and mAP of 58.7%, showing substantial improvements over existing semi-supervised methods.
  • Ablation studies confirmed the effectiveness of each component (HEL, CRE, CCL), with the full model (B+CRE+CMCL+CLAE) achieving the best results.

Limitations and Future Work

  • Limitations: While the method significantly reduces annotation costs, it still relies on intra-modal identity labels. The performance is also influenced by the quality of initial expert predictions, which can be affected by modality differences.
  • Future Work: The authors suggest that further improvements could be made through more advanced expert collaboration and adaptive learning strategies. The work opens avenues for developing more cost-effective and robust VIReID systems.

Summary updated

Abstract

To reduce the reliance of visible-infrared person re-identification (ReID) models on labeled cross-modal samples, this paper explores a weakly supervised cross-modal person ReID method that uses only single-modal sample identity labels, addressing scenarios where cross-modal identity labels are unavailable. To mitigate the impact of missing cross-modal labels on model performance, we propose a heterogeneous expert collaborative consistency learning framework, designed to establish robust cross-modal identity correspondences in a weakly supervised manner. This framework leverages labeled data from each modality to independently train dedicated classification experts. To associate cross-modal samples, these classification experts act as heterogeneous predictors, predicting the identities of samples from the other modality. To improve prediction accuracy, we design a cross-modal relationship fusion mechanism that effectively integrates predictions from different experts. Under the implicit supervision provided by cross-modal identity correspondences, collaborative and consistent learning among the experts is encouraged, significantly enhancing the model's ability to extract modality-invariant features and improve cross-modal identity recognition. Experimental results on two challenging datasets validate the effectiveness of the proposed method.
BibTeX
@inproceedings{zhang2025weakly,
  author = {Yafei Zhang and Lingqi Kong and Huafeng Li and Jie Wen},
  title = {{Weakly Supervised Visible-Infrared Person Re-Identification via Heterogeneous Expert Collaborative Consistency Learning}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2025},
}
Figure