Diverse Semantics-Guided Feature Alignment and Decoupling for Visible-Infrared Person Re-Identification

Neng Dong, Shuanglin Yan, Liyan Zhang, Jinhui Tang

Introduction

Visible-Infrared Person Re-Identification (VI-ReID) is a critical task in modern surveillance, enabling the tracking of individuals across heterogeneous camera networks that utilize standard visible-light (RGB) sensors during the day and infrared (IR) sensors at night. The core challenge of VI-ReID is twofold: the modality gap, where the same person appears fundamentally different across spectral ranges, and style noise, where environmental factors like lighting, shadows, and contrast overshadow identity-specific features.

While recent advancements have leveraged Large Language-Vision Models (LLVMs) like CLIP to bridge the modality gap using text as a common denominator, these methods often suffer from "semantic rigidity." By relying on fixed, repetitive prompt templates (e.g., "A photo of a person"), models tend to overfit to specific sentence structures rather than learning the underlying human characteristics. This paper introduces the Diverse Semantics-guided Feature Alignment and Decoupling (DSFAD) framework to overcome these limitations through diverse textual guidance and explicit feature separation.

Methodology

The DSFAD framework is built upon three primary modules designed to align cross-modality features while filtering out identity-irrelevant noise.

1. Diverse Semantics-guided Feature Alignment (DSFA)

To combat semantic rigidity, the DSFA module utilizes a two-step generation process. First, ChatGPT is employed to generate ten distinct sentence templates. Second, LLaVA (a vision-language model) populates these templates with fine-grained details regarding a person's appearance, such as clothing type, hair color, and accessories. By aligning visual features with this diverse array of descriptions, the model is forced to focus on high-level semantic concepts rather than memorizing fixed textual patterns. This approach treats language as a "universal translator" that remains invariant regardless of whether the input image is RGB or infrared.

2. Semantic Margin-guided Feature Decoupling (SMFD)

Environmental "style" often acts as a distractor in Re-ID tasks. The SMFD module employs Instance Normalization (IN) to decouple features into two distinct bins: identity-relevant features and style-related features. To ensure a clean separation, the authors implement a "margin loss." This loss function encourages identity features to cluster near their corresponding textual descriptions in the embedding space while simultaneously pushing style features—which should contain no identity information—away from the identity embeddings.

3. Semantic Consistency-guided Feature Restitution (SCFR)

Standard normalization techniques can be overly aggressive, occasionally discarding subtle identity-relevant details along with the style noise. The SCFR module acts as a recovery mechanism. It "excavates" any remaining identity-useful information from the style bin and restores it to the identity representation. This ensures that the final feature vector is both clean of noise and as discriminative as possible.

Experimental Evaluation

The DSFAD framework was evaluated on three benchmark datasets, each representing different challenges in the VI-ReID field:

  • SYSU-MM01: A large-scale dataset featuring 491 identities. DSFAD achieved 78.4% Rank-1 accuracy and 74.7% mAP, setting a new state-of-the-art (SOTA) benchmark and outperforming previous models like ARGN and HOS-Net.
  • LLCM: A challenging low-light cross-modality dataset where DSFAD reached 66.2% Rank-1 accuracy, demonstrating significant robustness in extreme lighting conditions.
  • RegDB: A standard dataset for paired visible and thermal images, where the model continued to show competitive performance.

A key finding from the ablation studies was that the "diversity" of the text descriptions was a primary driver of performance; using varied templates nearly doubled the accuracy gains compared to using a single, fixed-structure description.

Discussion and Future Work

The success of DSFAD highlights that the structure of language guidance is as important as the content. By treating style as a distraction and explicitly decoupling it from identity, the model achieves a more generalized understanding of human appearance.

However, some limitations remain. The current method of randomly selecting sentence templates can introduce instability during the training phase. Furthermore, the reliance on LLaVA means that errors in the generated descriptions (especially in low-quality infrared images) can propagate through the model. Future research will explore adaptive template selection and more dynamic, attribute-based "concept sets" to further refine the semantic guidance for pedestrian identification.

Summary updated

BibTeX
@article{dong2025diverse,
  author = {Neng Dong and Shuanglin Yan and Liyan Zhang and Jinhui Tang},
  title = {{Diverse Semantics-Guided Feature Alignment and Decoupling for Visible-Infrared Person Re-Identification}},
  journal = {IEEE Trans. Inf. Forensics Secur.},
  year = {2025},
  doi = {10.1109/tifs.2025.3629581},
}
Figure