Abstract
In the field of intelligent surveillance, person re-identification (Re-ID) is a fundamental task for matching individuals across non-overlapping camera views. While visible-light matching has achieved high performance, cross-modality matching between visible and infrared (IR) images remains a significant challenge. This paper presents a novel approach using single-modality self-supervised information mining to bridge the "modality gap" and overcome the scarcity of labeled cross-modal data. By leveraging abundant labeled visible-light data, the proposed method extracts spectrum-invariant features to guide the learning of cross-modal matching models.
The Challenge: Bridging the Modality Gap
Matching a person's identity between standard color images and low-light infrared images is difficult due to two primary hurdles:
- Visual Discrepancy: Significant differences in spectral imaging between visible and infrared light create a wide modality gap. For instance, a red shirt in daylight looks completely different in a thermal scan.
- Data Scarcity: Manually labeling cross-modality identities is labor-intensive and often difficult for human annotators, leading to a lack of high-quality training data.
Methodology: Mining Spectrum-Invariant Features
To address these issues, the researchers introduced a two-pronged strategy focused on extracting features that remain constant regardless of the camera type.
Random Single-Channel Mask (RSCM)
The authors developed RSCM, a data augmentation technique that randomly masks two of the three RGB channels in visible-light images. By forcing the model to recognize individuals using only a single channel (mimicking the single-channel nature of IR), the network learns to prioritize structural and textural cues—such as clothing patterns and body shape—over specific color values.
Dual-Model Mutual Learning
The framework employs two models trained simultaneously:
- Three-channel model: Processes full RGB data.
- Single-channel model: Processes masked data. Through a mutual learning loss (KL divergence), these models exchange knowledge. The single-channel model learns identity-rich features from the RGB model, while the RGB model becomes more robust by learning to ignore color-specific noise that does not translate across modalities.
Two-Stage Training Strategy
The process begins with pre-training on large-scale labeled RGB data (including synthetic datasets like UnrealPerson) to mine robust cross-spectral self-supervised information. This is followed by supervised fine-tuning on specific cross-modality datasets to bridge the final domain gap between RGB and IR.
Experimental Results
The proposed method was evaluated on three major benchmarks: SYSU-MM01, RGBNT201, and RegDB.
- RGBNT201: The approach achieved a significant performance boost, with the mean Average Precision (mAP) increasing by nearly 5% over previous state-of-the-art methods.
- SYSU-MM01: In the "All Search" mode, the model reached a Rank-1 accuracy of 73.0% and an mAP of 70.3%, outperforming more complex specialized architectures.
- RegDB: The system achieved 89.9% Rank-1 accuracy for visible-to-thermal matching tasks.
Conclusion
By shifting the focus from color-dependent features to spectrum-invariant structural information, this research provides a scalable solution to the cross-modality Re-ID problem. The use of synthetic data and self-supervised mining effectively reduces the reliance on expensive manual cross-modal labeling, paving the way for more robust surveillance systems capable of operating across diverse lighting conditions.
Summary updated