Adapting Local Features for Face Detection in Thermal Image

Chao Ma, Trung Ngo Thanh, Hideaki Uchiyama, Hajime Nagahara, Atsushi Shimada, Rin-Ichiro Taniguchi

TL;DR

This paper tackles the challenge of face detection in thermal images by proposing novel local features that leverage thermal properties and a mixed-feature AdaBoost training approach for cascade classifiers. The work demonstrates improved face detection performance in thermal imagery compared to existing methods, offering a more robust solution for scenarios where visible light cameras struggle.

What Problem Does This Paper Address?

Face detection is a fundamental task in computer vision, but traditional methods relying on visible light (RGB) cameras face several limitations. These include sensitivity to lighting conditions, issues with makeup, and the inability to distinguish real faces from printed ones. Thermal cameras offer an alternative by capturing temperature distributions, which are less affected by lighting and can inherently differentiate printed faces. However, existing face detection techniques, often designed for visible light, are not optimally adapted for the unique characteristics of thermal images. Specifically, there's a lack of local features that effectively exploit the constant temperature distribution of human faces in thermal imagery, leading to suboptimal performance.

Methods

The authors introduce two main contributions:

  • Novel Local Features: They extend the Multi-Block Local Binary Pattern (MB-LBP) feature.
  • Multi-Block Local Ternary Pattern (MB-LTP): This feature introduces a margin around the reference pixel temperature to improve robustness against thermal camera noise.
  • Absolute Multi-Block LBP (AMB-LBP) and Absolute Multi-Block LTP (AMB-LTP): These features leverage the generally constant facial temperature distribution by using an absolute temperature reference, optimized by AdaBoost, to enhance discriminative power.
  • Mixed-Feature AdaBoost Training: They propose an AdaBoost-based training method to build cascade classifiers that incorporate multiple types of local features (Haar-like, HOG, and their newly proposed features). This "mixed feature pool" aims to enhance the descriptive power of the cascade classifier by combining features with different strengths.

Datasets

  • Custom Dataset: The authors captured a new dataset for their experiments. This dataset consists of 8400 thermal images from 20 participants (14 males, 6 females) across 10 distances, 2 appearances (with/without glasses), and 21 poses.
  • Existing Datasets (Mentioned): UCHThermalFace and NVIE are mentioned as existing thermal face datasets, but they are noted to contain grayscale images that lose temperature information, making them unsuitable for this research.

Key Insights and Ideas

  • Thermal Signatures are Advantageous: The inherent temperature distribution of faces provides a more stable and less lighting-dependent cue than visible light, making thermal imaging a promising modality for face detection.
  • Absolute Temperature Matters: Simply comparing local pixel temperatures (like in MB-LBP/LTP) isn't enough. Using an absolute temperature reference (AMB-LBP/LTP) significantly improves the ability to distinguish faces from other objects, especially when environmental temperatures vary.
  • Margins Improve Robustness: Introducing a margin in feature calculations (MB-LTP) helps mitigate the impact of thermal camera noise, leading to more reliable feature responses.
  • Mixed Features Boost Performance: Combining different types of features (Haar-like, HOG, and novel thermal features) in a single cascade classifier leads to superior performance compared to using any single feature type. This highlights the benefit of diverse descriptive power.
  • AMB-LTP is the Top Performer: The combination of considering absolute temperature and a margin (AMB-LTP) within a mixed-feature cascade classifier yielded the best results.

Key Results

  • Improved Performance: The proposed methods (novel features and mixed-feature training) effectively improved face detection performance in thermal images.
  • Best Performing Combination: The cascade classifier trained with a mixed feature pool of {Haar-like Features, HOG Features, AMB-LTP Features} achieved the best performance.
  • Real-World Scenarios: In a field experiment with various environmental conditions (temperature, humidity, lighting), thermal images consistently showed an advantage over RGB images for face detection, especially under challenging lighting or when dealing with printed faces.
  • Privacy Advantage: Thermal images inherently offer better privacy protection as facial identities are harder to discern, eliminating the need for blurring in many applications.

Limitations and Future Work

  • Dataset Size: The authors intend to increase the number of samples and add more variations to their custom dataset in the future.
  • CNN Exploration: They plan to explore training Convolutional Neural Network (CNN) models using their dataset as a future research direction.
  • Computational Cost: While performance is improved, the use of multiple feature types in cascade classifiers can increase the overall processing time due to the overhead of managing different feature calculations. However, the authors note that their proposed features (AMB-LBP/LTP) can lead to shorter detection times due to better rejection ability.

Summary updated

BibTeX
@inproceedings{ma2017adapting,
  author = {Chao Ma and Trung Ngo Thanh and Hideaki Uchiyama and Hajime Nagahara and Atsushi Shimada and Rin-Ichiro Taniguchi},
  title = {{Adapting Local Features for Face Detection in Thermal Image}},
  booktitle = {Sensors},
  year = {2017},
  url = {https://openreview.net/forum?id=C6tO1xI64_},
}
Figure