Evaluating and Enhancing Face Anti-Spoofing Algorithms for Light Makeup: A General Detection Approach

Zhimao Lai, Yang Guo, Yongjian Hu, Wenkang Su, Renhai FengSensors 2024 · DOI

TL;DR

This paper addresses the challenge of face anti-spoofing systems misidentifying lightly made-up faces as fakes, a common issue due to a lack of relevant datasets and algorithms. The authors introduce a new dataset for light makeup faces and propose a novel anti-spoofing algorithm that significantly improves detection performance in these scenarios.

What Problem Does This Paper Address?

Face recognition systems are increasingly ubiquitous, but they remain vulnerable to "spoofing" attacks, where a fake face (e.g., a photo or mask) is presented to fool the system. Face anti-spoofing (FAS) algorithms are designed to detect these attacks. However, a significant gap exists in current FAS research: the impact of everyday light makeup on these systems. While heavy makeup might be considered an attack vector, most people wear light makeup daily, which subtly alters facial appearance. Existing FAS algorithms, often trained on bare faces or heavily altered faces, struggle with this common scenario, leading to potential user inconvenience if legitimate users are flagged as fakes. This paper aims to bridge this gap by developing a robust FAS approach specifically for light makeup.

Methods

The authors propose a novel general face anti-spoofing algorithm designed for light makeup. Key components include:

  • Makeup Augmentation Module: This module synthetically generates light makeup effects on existing bare-faced images to create more diverse training data, simulating real-world variations. It uses Thin Plate Spline (TPS) transformation and Poisson blending, with careful screening of input images to minimize artifacts.
  • Batch Channel Normalization Module: This module enhances the standard normalization layers in the backbone network (an improved ResNet-18) to better adapt to diverse datasets and improve generalization.
  • Backbone Network with EMA: The backbone network is updated using both gradient descent and Exponential Moving Average (EMA), aiming to stabilize feature extraction.
  • Asymmetric Virtual Triplet Loss Module: This loss function is designed to address class imbalance by making genuine (real) faces more compact in the feature space and fake faces more dispersed, improving differentiation.
  • Nearest Neighbor Supervised Contrastive Module: This module improves upon standard contrastive learning by incorporating neighbor information to better capture sample distributions and relationships, leading to more robust feature representations.

The final loss function is a weighted combination of global cross-entropy loss, the asymmetric virtual triplet loss, and the nearest neighbor supervised contrastive loss.

Datasets

  • New Dataset (MkX): The authors construct a novel face anti-spoofing database that specifically includes light makeup faces. This dataset is created by applying makeup transfer algorithms (SpMT and EleGANt) to real, bare-faced images from existing public databases (MSU-MFSD, Replay-Attack, CASIA-FASD, OULU-NPU). A criterion for determining "light makeup" based on confidence scores from a face comparison model (Face++) is introduced to select appropriate data.
  • Existing Datasets: The paper also utilizes standard face anti-spoofing datasets (M, R, C, O) for training and cross-database evaluation.

Key Insights and Ideas

  • Light Makeup is a Significant Challenge: The study confirms that existing state-of-the-art FAS algorithms experience a notable performance drop when faced with light makeup, highlighting a critical blind spot in current research.
  • Synthetic Data Generation is Crucial: The proposed makeup augmentation module effectively simulates light makeup, creating a more representative dataset and improving model robustness.
  • Beyond Standard Contrastive Learning: The nearest neighbor supervised contrastive learning approach offers a more nuanced way to learn feature representations by considering sample relationships, leading to better separation of genuine and spoofing samples.
  • Asymmetric Loss for Better Separation: The asymmetric virtual triplet loss is key to handling the inherent differences in feature distribution between genuine and fake faces, particularly in the context of subtle variations like light makeup.

Key Results

  • The proposed algorithm achieves the best average AUC (92.21%) and HTER (13.82%) on light makeup faces across various testing strategies, significantly outperforming existing methods.
  • On the ICM_Mk(O) dataset, the proposed method achieved an AUC of 91.16% and an HTER of 15.56%, demonstrating strong performance even in challenging cross-domain scenarios with light makeup.
  • Ablation studies show that each proposed module (makeup augmentation, batch channel normalization, asymmetric virtual triplet loss, and nearest neighbor supervised contrastive learning) contributes significantly to the overall performance improvement.
  • The algorithm demonstrates good generalization capabilities on datasets without light makeup, maintaining an AUC above 96% and HTER below 10%.

Limitations and Future Work

  • Dataset Expansion: The authors plan to expand the database to include more diverse user profiles, such as individuals wearing glasses or hats, to better simulate real-world conditions.
  • Handling Accessories: Future work will focus on developing features that are less sensitive to variations caused by makeup or accessories, improving robustness across diverse environments.
  • Practical Applications: The research aims to explore practical implications in settings like TSA security lines, where accurate recognition of individuals with accessories is crucial.
  • Refining Criteria: Further refinement of the light makeup determination criteria is also planned.

Summary updated

Abstract

Makeup modifies facial textures and colors, impacting the precision of face anti-spoofing systems. Many individuals opt for light makeup in their daily lives, which generally does not hinder face identity recognition. However, current research in face anti-spoofing often neglects the influence of light makeup on facial feature recognition, notably the absence of publicly accessible datasets featuring light makeup faces. If these instances are incorrectly flagged as fraudulent by face anti-spoofing systems, it could lead to user inconvenience. In response, we develop a face anti-spoofing database that includes light makeup faces and establishes a criterion for determining light makeup to select appropriate data. Building on this foundation, we assess multiple established face anti-spoofing algorithms using the newly created database. Our findings reveal that the majority of these algorithms experience a decrease in performance when faced with light makeup faces. Consequently, this paper introduces a general face anti-spoofing algorithm specifically designed for light makeup faces, which includes a makeup augmentation module, a batch channel normalization module, a backbone network updated via the Exponential Moving Average (EMA) method, an asymmetric virtual triplet loss module, and a nearest neighbor supervised contrastive module. The experimental outcomes confirm that the proposed algorithm exhibits superior detection capabilities when handling light makeup faces.
BibTeX
@inproceedings{lai2024evaluating,
  author = {Zhimao Lai and Yang Guo and Yongjian Hu and Wenkang Su and Renhai Feng},
  title = {{Evaluating and Enhancing Face Anti-Spoofing Algorithms for Light Makeup: A General Detection Approach}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24248075},
}
Figure