Abstract
Visible-Infrared Person Re-Identification (VIReID) is a critical task for 24-hour surveillance, requiring the matching of individuals across color-rich visible light cameras and monochromatic infrared cameras. The primary challenge lies in the significant modality gap. We propose the Richer Semantics, Better Alignment (RSBA) framework, which leverages a large vision-language model (LLaVA) to generate explicit, detailed textual descriptions for pedestrian images. By aligning visual features with these enriched semantics and employing an indirect matching strategy to handle cross-modality color discrepancies, RSBA achieves state-of-the-art performance on major benchmarks including SYSU-MM01, RegDB, and LLCM.
Introduction
Visible-Infrared Person Re-Identification (VIReID) involves matching the same person across different camera types—standard visible light cameras used during the day and infrared cameras used at night. The core difficulty is the modality gap: visible images contain rich color information, whereas infrared images are monochromatic and reflect thermal properties.
Most existing methods attempt to learn a shared feature space using only image data, which is difficult due to the disparate visual distributions. Recent approaches have integrated "learnable textual prompts" (similar to CLIP), but these often act as "black boxes"—they are coarse and lack the specific, explicit semantic details (such as "braided hair" or "carrying a shoulder bag") necessary to distinguish between similar-looking individuals.
Methodology
The RSBA framework introduces three key modules to bridge the modality gap through explicit semantics:
1. Explicit Semantics-Guided Feature Alignment (ESFA)
Instead of relying on generic, learnable prompts, RSBA utilizes LLaVA (a large language-vision model) to generate detailed natural language descriptions for every training image. The model learns to align visual features with these explicit textual descriptions, forcing the visual encoder to attend to specific, identifiable attributes.
2. Consistent Similarity-Guided Indirect Alignment (CSIA)
A significant hurdle in VIReID is that text descriptions can conflict across modalities; for instance, a "blue skirt" in visible light appears "gray" in infrared. Direct alignment would confuse the model. CSIA implements indirect alignment: it ensures that the similarity between an image and its corresponding text is consistent with the similarity between the texts of the two different modalities. This creates a semantic bridge without forcing infrared features to match color-specific terms directly.
3. Cross-View Semantics Compensation (CVSC)
Since a single camera view may only capture partial information (e.g., only the front or back), CVSC uses an attention mechanism to fuse descriptions from different views of the same person. This "one-to-many" matching scheme provides a more complete and robust semantic profile.
Importantly, the text generation and fusion components are only required during training. During inference, the model relies solely on the visual encoder, ensuring high efficiency for real-world deployment.
Experimental Results
The RSBA framework was evaluated on three major VIReID benchmarks, consistently outperforming previous state-of-the-art (SOTA) methods:
- SYSU-MM01: Achieved 78.4% Rank-1 accuracy and 74.8% mAP in all-search single-shot mode.
- RegDB: Reached 95.2% Rank-1 accuracy for visible-to-infrared search.
- LLCM: Achieved 66.0% Rank-1 accuracy, demonstrating robustness in challenging low-light environments.
Ablation studies confirmed that each module (ESFA, CSIA, and CVSC) contributes significantly to the overall performance gains.
Conclusion
The RSBA framework demonstrates that detailed, human-readable descriptions from large vision-language models provide a far more effective "anchor" for feature alignment than abstract learnable vectors. By acknowledging and accounting for modality-specific differences through indirect alignment and multi-view fusion, RSBA sets a new standard for visible-infrared person re-identification. Future work will focus on improving text quality and exploring more diverse methods for acquiring pedestrian semantics.
Summary updated