TL;DR
This paper introduces a novel weakly supervised approach for Visible-Infrared Person Re-Identification (VIReID) that eliminates the need for costly cross-modal identity labels. By training modality-specific "experts" and fostering their collaborative learning, the method effectively establishes cross-modal identity correspondences, achieving performance comparable to fully supervised methods.
What Problem Does This Paper Address?
Visible-Infrared Person Re-Identification (VIReID) is crucial for surveillance and smart city applications, aiming to match pedestrians across visible and infrared camera feeds. The primary challenge is the significant visual discrepancy between these modalities, which makes annotating cross-modal datasets extremely expensive and labor-intensive. Existing methods often rely on fully labeled cross-modal data, or struggle with noisy pseudo-labels in semi-supervised and unsupervised settings. This paper tackles the problem of VIReID under a weakly supervised setting, where only single-modality identity labels are available, thus drastically reducing annotation costs and complexity.
Methods
The core of the proposed method is a Heterogeneous Expert Collaborative Consistency Learning (HEL-CCL) framework. It comprises three main stages:
- Heterogeneous Expert Learning (HEL): Separate classification experts (based on ResNet-50 encoders) are trained independently for visible and infrared modalities using only their respective intra-modal identity labels. These experts learn to extract identity-discriminative features within their own modality.
- Cross-Modal Relationship Establishment (CRE): The trained experts are used to predict identities for samples from the other modality. By analyzing the consistency and agreement between these predictions, the framework establishes robust cross-modal identity correspondences. This is achieved by categorizing matches into consistent, unique, and conflicting, and constructing matrices (Mc, Ms, Mw) to represent these relationships.
- Collaborative Consistency Learning (CCL): This stage leverages the established cross-modal correspondences to guide the training of the feature encoders and a shared classifier.
- Cross-Modal Consistency Learning (CMCL): A cross-entropy loss is applied to enforce consistent predictions for samples with established correspondences. A relaxed loss is used for uncertain correspondences to mitigate label noise.
- Collaborative Learning Among Experts (CLAE): To further enhance consistency, a collaborative loss encourages experts to make similar predictions for cross-modal positive samples. Prototypes for each identity are used to facilitate this. The strength of this constraint is dynamically adjusted based on the confidence of expert predictions.
This iterative process of experts learning, establishing correspondences, and then collaboratively refining features leads to modality-invariant representations.
Datasets
The method was evaluated on two widely used VIReID datasets:
- SYSU-MM01: A large-scale dataset with images from four visible and two near-infrared cameras, covering diverse indoor and outdoor scenes.
- LLCM: A challenging dataset captured in low-light environments using nine visible and nine infrared cameras.
Key Insights and Ideas
- Weak Supervision is Sufficient: The paper demonstrates that VIReID can be effectively performed with only single-modality identity labels, significantly reducing the annotation burden.
- Expert Collaboration is Key: By treating modality-specific classifiers as "heterogeneous experts" and forcing them to collaborate and agree on cross-modal identities, robust correspondences can be learned even without direct supervision.
- Leveraging Uncertainty: The method intelligently handles uncertain cross-modal correspondences by using a relaxed loss, preventing noisy labels from degrading performance.
- Modality Discrepancy Mitigation: The collaborative consistency learning framework effectively bridges the gap between visible and infrared features, leading to more unified representations.
Key Results
The proposed weakly supervised method achieved performance comparable to, and in some cases surpassing, state-of-the-art fully supervised methods on both SYSU-MM01 and LLCM datasets.
- On SYSU-MM01 (all-search), the method achieved Rank-1 accuracy of 70.4% and mAP of 66.6%, outperforming several supervised methods and significantly outperforming semi-supervised and unsupervised approaches.
- On LLCM (VIS to IR), it achieved Rank-1 accuracy of 55.3% and mAP of 58.7%, showing substantial improvements over existing semi-supervised methods.
- Ablation studies confirmed the effectiveness of each component (HEL, CRE, CCL), with the full model (B+CRE+CMCL+CLAE) achieving the best results.
Limitations and Future Work
- Limitations: While the method significantly reduces annotation costs, it still relies on intra-modal identity labels. The performance is also influenced by the quality of initial expert predictions, which can be affected by modality differences.
- Future Work: The authors suggest that further improvements could be made through more advanced expert collaboration and adaptive learning strategies. The work opens avenues for developing more cost-effective and robust VIReID systems.
Summary updated