Implicit Modality Knowledge Alignment and Uncertainty Estimation for visible-infrared person re-identification

Song Wu, Shihao Shan, Guoqiang Xiao, Michael S. Lew, Xinbo Gao

Introduction

Visible-Infrared Person Re-identification (VI-ReID) is a critical task in modern surveillance, enabling the tracking of individuals across cameras that switch between visible (RGB) and infrared (IR) modes. The primary challenge lies in the "modality gap"—the significant stylistic and informational difference between color-rich RGB images and texture-focused thermal IR images. Traditional approaches often rely on computationally heavy Generative Adversarial Networks (GANs) to bridge this gap, which can be unstable and resource-intensive. Furthermore, standard classification models often suffer from overconfidence, leading to high-probability errors.

The IMKA-UE Framework

The proposed IMKA-UE framework addresses these issues through two innovative modules: Implicit Modality Knowledge Alignment (IMKA) and Uncertainty Estimation (UE).

Implicit Modality Knowledge Alignment (IMKA)

Instead of generating full-scale images, the IMKA module focuses on feature-level augmentation.

  • Implicit Modality Generation (IMG): A lightweight two-layer convolutional module extracts supplementary identity information to create "implicit" samples. These samples serve as a middle ground between the visible and infrared modalities.
  • Significant Knowledge Alignment (SKA): To ensure these generated features are meaningful, a distribution consistency loss ($L_{dcon}$) is applied. This aligns the implicit features within a common embedding space, preventing the accumulation of noise.
Conceptual overview of IMKA-UE: (a) Implicit samples bridging the modality gap and (b) uncertainty estimation correcting misclassifications.

Uncertainty Estimation (UE)

To combat overconfidence, the framework incorporates a Dirichlet-based uncertainty estimation strategy. By treating classifier outputs as "evidence" rather than simple probabilities, the model can quantify its own uncertainty. An evidence loss ($L_{evi}$) penalizes high-confidence incorrect predictions, effectively pushing misclassified samples toward their correct identity clusters during the training process.

Experimental Evaluation

The framework was rigorously tested on two primary benchmarks:

  • SYSU-MM01: On the "All Search" mode, IMKA-UE achieved a 71.94% Rank-1 accuracy and 67.69% mAP, setting a new state-of-the-art by outperforming previous methods like DCLNet.
  • RegDB: The model reached a high of 92.59% Rank-1 accuracy in the Visible-to-Infrared task, a 3.47% improvement over the previous best (FMCNet).

The use of the mean Inverse Negative Penalty (mINP) metric further highlighted the model's robustness, showing significant gains in retrieving "hard" matches (e.g., 65.79% on RegDB).

Conclusion

IMKA-UE demonstrates that effective modality alignment does not require complex generative architectures. By leveraging lightweight implicit samples and a principled approach to uncertainty, the framework achieves superior performance and robustness in cross-modal person re-identification. Future research will focus on optimizing training efficiency and exploring the model's resilience to noisy label environments.

Summary updated

Abstract

Highlights•A IMKA method is proposed to reduce the large semantic domain gap in VI-ReID.•The IMKA can learn valuable supplementary information from original modalities.•The IMKA can align the distribution of learned significant knowledge.•The UE can quantify the confidence level of prediction in a Dirichlet space.•The evaluation results show that our IMKA-UE achieves SOTA performance.
BibTeX
@article{wu2025implicit,
  author = {Song Wu and Shihao Shan and Guoqiang Xiao and Michael S. Lew and Xinbo Gao},
  title = {{Implicit Modality Knowledge Alignment and Uncertainty Estimation for visible-infrared person re-identification}},
  journal = {Expert Systems with Applications},
  year = {2025},
  doi = {10.1016/j.eswa.2024.125291},
}
Figure