Multi-Domain Biometric Recognition using Body Embeddings

Anirudh Nanduri, Siyuan Huang, Rama Chellappa

Multi-Domain Biometric Recognition using Body Embeddings

Introduction

Biometric identification has reached high levels of maturity in the visible (VIS) spectrum. However, real-world security applications often demand performance in challenging environments—total darkness, fog, or across long distances—where visible light cameras are ineffective. While Near-Infrared (NIR) and Short-Wave Infrared (SWIR) sensors offer imagery similar to visible light, Medium-Wave (MWIR) and Long-Wave (LWIR) sensors capture thermal radiation. In these thermal domains, traditional facial features often lose their distinctiveness due to low resolution and the inherent nature of heat signatures.

This research addresses the challenge of multi-domain recognition, matching identities across VIS, SWIR, MWIR, and LWIR simultaneously. The central thesis is that in thermal environments, we should shift our focus from facial features to the entire body.

Proposed Methodology

The authors propose a robust pipeline centered on a Vision Transformer (ViT) backbone designed to extract multi-granular features.

The proposed multi-domain biometric recognition pipeline utilizing a ViT backbone for global and local semantic feature extraction.

The architecture employs several key strategies:

  • Multi-Granular Feature Extraction: Beyond a global image "fingerprint," the model utilizes "local semantic tokens." Using anatomically-aware keypoint prompts, it identifies and extracts features from specific regions: the face, torso, and lower body.
  • Feature Fusion: These local features are averaged and concatenated with a global token, ensuring the final embedding captures both the overall silhouette and fine-grained regional details.
  • Domain-Aware Training: The model is trained using batches containing images of the same subject across all four spectral domains. A combination of Cross-Entropy (Identity) and Triplet Loss is used to minimize the distance between different spectral representations of the same individual.

Experimental Results

The model was evaluated on two primary benchmarks: the IJB-MDF (IARPA Janus Benchmark Multi-Domain Face) and the LLCM (Low-Light Cross-Modal) dataset.

A critical finding of the study is the superiority of body embeddings over face embeddings in thermal domains. Traditional face models, such as AdaFace, struggle significantly when transitioning from visible light to thermal IR. In contrast, the proposed body-centric model maintains high accuracy.

Comparison of Rank-1 accuracy between face-based (AdaFace) and body-based models across different infrared domains.

As shown in the results, while a pretrained face model (AdaFace) achieves 0.00% accuracy in MWIR, the finetuned body model achieves a consistent 77.78% Rank-1 accuracy across SWIR, MWIR, and LWIR domains when matched against VIS gallery images. On the LLCM dataset, the model achieved state-of-the-art mAP scores of 72.9% (VIS-to-IR) and 70.3% (IR-to-VIS).

Key Insights

  • Thermal Consistency: The research observed high consistency between MWIR and LWIR features. A model trained on one generalizes well to the other, suggesting a shared "thermal feature space" distinct from the "reflective feature space" of VIS and SWIR.
  • The Power of VIS Pre-training: A ViT pre-trained on large-scale visible light data (e.g., LUPerson) can be effectively adapted to infrared domains through simple fine-tuning, provided the focus remains on body semantics.
  • Ablation Analysis: Removing any local semantic feature (face, torso, or legs) resulted in a performance drop, particularly in the LWIR domain, confirming that holistic body analysis is vital for thermal biometrics.

Conclusion

By leveraging the structural consistency of the human body across different spectral bands, this work demonstrates a significant leap in multi-domain biometric recognition. Moving beyond the face allows for robust identification even in the "blurry" world of thermal imaging, paving the way for more reliable security systems in adverse conditions.

Summary updated

BibTeX
@inproceedings{nanduri2025multi,
  author = {Anirudh Nanduri and Siyuan Huang and Rama Chellappa},
  title = {{Multi-Domain Biometric Recognition using Body Embeddings}},
  booktitle = {FG},
  year = {2025},
  doi = {10.1109/fg61629.2025.11099075},
}
Figure