Introduction
Heterogeneous Face Recognition (HFR) is a critical technology for low-light environments, such as mobile face unlocking and border security. It requires matching probe images captured in the near-infrared (NIR) spectrum against a gallery of standard visible-light (VIS) photographs. Historically, the research community has operated under the assumption that a massive "domain gap" exists between these two modalities, necessitating complex solutions like image-to-image translation or intricate feature-level alignment.
However, this paper challenges that narrative. The authors argue that the perceived domain gap was largely a byproduct of using small-capacity models. By investigating the capabilities of modern, large-scale neural networks, they reveal that the richness of VIS datasets can be directly leveraged to solve the NIR problem with surprisingly simple tools.
The Physics-Inspired Bridge
A core insight of this work is the spectral relationship between visible light and near-infrared radiation. Standard RGB sensors capture light in three bands: Blue, Green, and Red. As shown in the spectral sensitivity curves, the Red channel has the highest spectral overlap with the NIR spectrum.
By recognizing this "Physics Bridge," the authors propose Red Channel Augmentation. During the pre-training phase on massive VIS datasets (like MS1Mv3), they randomly replace RGB images with versions where all three channels are copies of the Red channel. This simple augmentation forces the model to learn features that are invariant to the color shifts and lack of chromatic information characteristic of NIR sensors, effectively preparing the model for the NIR domain before it ever sees an actual NIR image.
Proposed Framework: Regularized Subspace Fine-tuning
While large models exhibit remarkable "zero-shot" NIR capabilities, fine-tuning on specific NIR datasets is still necessary to reach peak performance. However, standard fine-tuning on small HFR datasets often leads to "catastrophic forgetting," where the model overfits to the small target set and loses the robust facial representations learned during VIS pre-training.
To address this, the authors introduce a two-stage framework:
- Enhanced VIS Pre-training: Utilizing the Red Channel Augmentation described above.
- Regularized Subspace Fine-tuning: This stage employs two key strategies to preserve the pre-trained embedding space. First, they use a Subspace Classifier, initializing weights with mean identity embeddings rather than random values. Second, they apply a Regularized Loss by continuing to feed VIS data into the model during NIR fine-tuning. This ensures the model maintains its general facial recognition intelligence while adapting to the specific sensor noise of the NIR data.
Key Results and Insights
The experimental results validate the authors' hypothesis that model scale significantly impacts the domain gap. While smaller models (e.g., MFN) show a drastic performance drop when moving from VIS to NIR, larger models like ResNet-100 maintain over 99% accuracy (at FAR=$10^{-4}$) on benchmarks like CASIA 2.0 and BUAA-VisNir, even in a zero-shot setting.
Furthermore, the proposed regularized fine-tuning approach set new state-of-the-art records across all major HFR benchmarks. On the high-quality LAMP-HQ dataset, the method achieved 99.35% accuracy, outperforming far more complex generative and domain-invariant models. Crucially, these models demonstrated superior cross-dataset robustness, proving they learned generalized facial features rather than just overfitting to specific sensor characteristics.
Conclusion
This research shifts the focus of HFR from complex cross-modal bridging to leveraging the power of scale and simple physical priors. By rethinking the domain gap, the authors demonstrate that the "NIR problem" is largely solvable through high-capacity models and intelligent regularization. Future work will explore how to distill this high-level performance into the smaller, more efficient models required for real-time mobile applications.
Summary updated