Introduction
Person Re-Identification (ReID) is a critical computer vision task focused on identifying the same individual across non-overlapping camera views. While traditional ReID relies on visible-light (RGB) images, real-world surveillance often operates across varying lighting conditions, leading to the development of Visible-Infrared ReID (VI-ReID). However, existing VI-ReID research typically assumes a "bi-modality mutual retrieval" setting—searching for an infrared (IR) query in an RGB gallery, or vice versa.
This paper argues that this assumption is unrealistic. In practice, a surveillance database likely contains a mixture of both visible and infrared images captured at different times. The authors propose a more practical paradigm: Mix-Modality Person Re-Identification (MM-ReID), where both the query and gallery sets contain a blend of modalities.
The Challenge: Modality Confusion
The primary obstacle in MM-ReID is Modality Confusion. In a mixed-modality gallery, models often prioritize modality features over identity features. For example, a model might find two different people in infrared images more similar to each other than the same person appearing in one infrared and one visible image. This leads to a ranklist where same-modality incorrect matches appear before cross-modality correct matches.
Proposed Methodology
To address these issues, the authors introduce two key innovations:
1. Cross-Identity Discrimination Harmonization Loss (CIDHL)
Standard triplet losses can be noisy when dealing with cross-modality data. CIDHL shifts the focus from individual samples to identity centers—the mean feature representation of an identity within a specific modality. By minimizing the distance between centers of the same identity across modalities and maximizing the distance between different identities, CIDHL provides a stable training signal. Additionally, a "harmonization" term ensures that samples of the same modality and identity cluster tightly around their respective centers, improving the model's discriminative power.
2. Modality Bridge Similarity Optimization Strategy (MBSOS)
MBSOS is a post-processing retrieval technique that treats the modality gap as a pathfinding problem. Instead of calculating a direct distance between a visible query and an infrared target, MBSOS identifies "bridge samples" within the gallery that share the same modality as the query. By using these intermediate samples as "stepping stones," the strategy effectively navigates the modality gap, resulting in a more accurate similarity metric during the testing phase.
Experimental Results
The proposed methods were evaluated on three major benchmarks: SYSU-MM01, RegDB, and LLCM, adapted for the MM-ReID paradigm.
The integration of CIDHL and MBSOS into state-of-the-art baselines (like AGW and DEEN) yielded significant performance improvements:
- SYSU-MM01: The AGW baseline saw a Rank-1 accuracy boost of 18.86% and an mAP increase of 13.76%.
- RegDB: Rank-1 accuracy improved by 6.09% with an mAP jump of 8.84%.
These results demonstrate that the proposed methods are highly effective and transferable across different model architectures, successfully mitigating the impact of modality confusion.
Conclusion and Future Directions
This work highlights the limitations of current bi-modality ReID research and establishes MM-ReID as a more practical and challenging benchmark. While CIDHL and MBSOS provide strong solutions, future work could explore more integrated training schemes for mixed data and more sophisticated "bridge" logic to further refine cross-modality retrieval in complex environments.
Summary updated