Face spoofing detection by fusing binocular depth and spatial pyramid coding micro-texture features

Xu Zhao, Xiao Song, Xu Zhao 0001, Tianwei LinICIP 2017 · DOI

TL;DR

This paper introduces two novel features, binocular depth and micro-texture, to combat increasingly sophisticated face spoofing attacks. By fusing these complementary features, the authors achieve state-of-the-art performance in distinguishing real faces from sophisticated fakes, demonstrating robustness across various attack types.

What Problem Does This Paper Address?

Face spoofing, where attackers present fake faces (photos, videos, 3D models) to fool facial recognition systems, is a significant security concern. Existing anti-spoofing methods often fall into categories like physiological sign analysis (e.g., blinking, mouth movements), texture analysis, or illumination analysis. However, these methods can be brittle, failing against high-quality fakes or requiring specific conditions. For instance, texture-based methods can be sensitive to illumination changes, while physiological methods might be fooled by advanced 3D models. This paper aims to develop more robust and generalizable anti-spoofing techniques by combining information from different modalities.

Methods

The core of this work lies in two novel feature extraction methods:

  • Template Face Matched Binocular Depth (TFBD) Feature: This method leverages a binocular camera setup to capture depth information. It first detects facial keypoints using the Constrained Local Models (CLM) approach. The crucial step is a "Template Face Registration" process. Here, the detected keypoints from the live face are iteratively aligned and transformed to match a pre-defined "Template Face" (derived from average keypoints of genuine faces). This registration process helps normalize depth variations due to pose and distance, making the depth feature more robust. The resulting 68-dimensional depth feature vector captures the 3D structure of the face.
  • Spatial Pyramid Coding Micro-Texture (SPMT) Feature: Recognizing that depth alone might not be sufficient, this method focuses on micro-texture. It first extracts micro-texture features using a Multi-Scale Local Binary Pattern (MSLBP) operator. These features are then encoded into a "Bag of Visual Words" (BOVW) representation. The novelty here is the "Spatial Pyramid Coding" algorithm. This algorithm partitions the BOVW representation into multiple spatial regions at different levels of resolution. It then computes weighted histograms of these regions and, importantly, "matching-degree vectors" that quantify how well the texture in each region matches pre-computed average "positive" (genuine) and "negative" (spoof) intra-class faces. These weighted histograms and matching-degree vectors are concatenated to form a high-dimensional SPMT feature vector.

Finally, the system employs a score-level fusion approach. Each of the TFBD and SPMT features is fed into a separate non-linear Support Vector Machine (SVM) classifier. The outputs (scores) from these two SVMs are then combined to make the final classification decision.

Datasets

The authors evaluated their methods on:

  • NUAA Photograph Imposter Database: A publicly available dataset.
  • CASIA-FASD Database: Another publicly available dataset.
  • A newly constructed dataset: This dataset comprises 6000 image pairs captured simultaneously by two fixed, calibrated web cameras. It includes variations in illumination, pose (head movements, spinning), distance, and various attack types (printed photos, photographic paper prints, iPad displays).

Key Insights and Ideas

  • Complementarity of Depth and Texture: The paper strongly advocates for combining 3D structural information (depth) with fine-grained surface details (micro-texture). Neither feature alone is sufficient for robust spoofing detection against all attacks.
  • Robustness through Registration: The Template Face Matched Binocular Depth feature's effectiveness is significantly boosted by the iterative registration process, which normalizes for pose and distance variations that would otherwise degrade depth-based cues.
  • Spatial Pyramid Coding for Texture: The novel spatial pyramid coding approach for texture features is crucial. By analyzing texture at different spatial granularities and comparing it against representative genuine and fake face models, it captures both local patterns and global structural context, making it resilient to variations.
  • Score-Level Fusion Efficiency: Using score-level fusion allows for independent training of classifiers for each feature type and is computationally efficient, as it combines decisions rather than raw features.

Key Results

  • High Accuracy: The fused TFBD and SPMT features achieved an impressive accuracy of 99.2% and an AUC of 0.998 on their custom dataset.
  • Performance Improvement: The TFBD feature alone showed a 5% improvement in accuracy over original depth features after template matching. The SPMT feature achieved nearly 95% accuracy.
  • Speed: The TFBD feature extraction runs at 50 fps, and the SPMT feature runs at 10 fps, making the overall system computationally feasible.
  • Robustness: The proposed method demonstrated strong performance against various attack types, including high-definition prints and videos, outperforming traditional methods.

Limitations and Future Work

  • Illumination Sensitivity of SPMT: While SPMT is more robust than some low-level texture descriptors, it can still be sensitive to intense illumination changes and high-definition prints.
  • Future Directions: The authors suggest that these robust features could be applied to other face recognition tasks beyond spoofing detection. The paper implies that further exploration of feature fusion strategies and potentially more advanced deep learning architectures could yield even better results.

Summary updated

BibTeX
@inproceedings{zhao2017face,
  author = {Xu Zhao and Xiao Song and Xu Zhao and Tianwei Lin},
  title = {{Face spoofing detection by fusing binocular depth and spatial pyramid coding micro-texture features}},
  booktitle = {ICIP},
  year = {2017},
  doi = {10.1109/icip.2017.8296250},
}
Figure