Introduction
Standard face recognition systems perform exceptionally well under controlled, visible-light conditions. However, their accuracy degrades significantly in "non-visual" domains—such as thermal (LWIR), short-wave infrared (SWIR), or long-range surveillance—where images are frequently blurry, noisy, or of low resolution.
Most existing solutions rely on Domain Adaptation, which requires massive labeled datasets for every specific environmental condition. This paper addresses the "single-source, multi-target" problem: adapting a model trained solely on visible light to work across multiple challenging domains without extensive retraining.
Methodology: Beyond Average Pooling
The researchers focus on template-based recognition, where a subject is represented by a collection of images or video frames rather than a single static shot. The traditional approach, "Average Pooling," treats all frames equally, allowing a single low-quality or "junk" frame to pollute the entire template and reduce accuracy.
To solve this, the authors propose two novel algorithms that weight images based on their inherent quality:
- Feature Norm as Quality Proxy: Modern face recognition models (like ArcFace or AdaFace) naturally produce feature vectors whose $L_2$ norm correlates with image quality. Clearer faces result in "longer" vectors, while degraded images produce "shorter" ones.
- Norm Pooling (NP): This method applies a
softmaxfunction to feature norms to calculate weights. High-quality images contribute more to the final template, while low-quality ones are suppressed. - Sparse Pooling (SP): Utilizing the
sparsemaxfunction, this variant can assign a weight of exactly zero to extremely poor frames. This allows the system to completely discard "junk" data that would otherwise degrade the template.
Experimental Results
The methods were evaluated using the IARPA JANUS Benchmark Multi-domain Face (IJB-MDF), a grueling dataset featuring visible and infrared imagery at distances up to 500 meters.
Key performance metrics (Rank-1 retrieval rates) include:
- Visible 500m: Norm Pooling achieved 86.01%, a significant improvement over the 81.12% baseline of Average Pooling.
- SWIR 30m: AdaFace combined with Norm Pooling reached 79.68%, outperforming standard averaging (77.29%).
Key Insights
- Inherent Quality Signals: The feature extractor itself signals quality through vector magnitude; no separate quality assessment model is required.
- Sparsity Benefits: Sparse Pooling proved highly effective, sometimes discarding over 70% of frames in a template to focus only on the highest-quality data.
- Model Synergy: Models trained with quality-aware loss functions, such as AdaFace, showed the strongest performance gains when paired with Norm Pooling.
Conclusion and Future Work
While these pooling methods significantly bridge the gap in SWIR and long-range visible domains, challenges remain in thermal (MWIR/LWIR) spectra where the domain gap is too vast for visible-light models. Future research will focus on automatically estimating the scaling parameters ($\lambda$) used in these algorithms to further generalize across diverse environments.
Summary updated