Partial Attack Supervision and Regional Weighted Inference for Masked Face Presentation Attack Detection

Meiling Fang, Fadi Boutros, Arjan Kuijper, Naser DamerarXiv 2021 · arXiv

TL;DR

The widespread use of face masks has severely degraded the accuracy of face Presentation Attack Detection (PAD) systems, particularly when dealing with "partial attacks" where a real mask is placed over a spoofed face. This paper introduces PAL-RW, a two-pronged solution that uses fine-grained Partial Attack Labeling (PAL) during training to correctly identify real mask regions as bona fide, combined with Regional Weighted Inference (RW) during testing to focus on the highly discriminative eye region. This architecture-agnostic approach significantly reduces error rates across multiple neural network backbones.

What Problem Does This Paper Address?

Face recognition and anti-spoofing systems (PAD) faced an unexpected challenge with the COVID-19 pandemic: widespread face mask adoption. Standard PAD algorithms often misclassify real, masked faces as presentation attacks (spoofs), or fail to catch sophisticated "partial attacks."

A partial attack occurs when an attacker places a physically real face mask over a printed photo or a digital screen displaying a fake face. Traditional pixel-wise supervision methods treat the entire attack image as a "zero map" (100% spoof). However, because the mask itself is a real object, forcing the network to learn that a real mask is an "attack" introduces significant noise during training. Furthermore, standard evaluation methods treat all facial regions equally during inference, ignoring the fact that when the lower face is covered, the eye region becomes the most critical area for verifying authenticity.

Methods

The authors propose PAL-RW, which consists of two main components that can be integrated into any pixel-wise PAD network architecture:

  1. Partial Attack Label (PAL) for Training: Instead of labeling the entire partial attack image as a spoof, the authors use 68 facial landmarks to segment the boundary of the face mask. Pixels corresponding to the real face mask are labeled as 1 (bona fide), while the uncovered, spoofed regions of the face are labeled as 0 (attack). This fine-grained labeling provides cleaner supervision and helps the model converge effectively.
Fig. 3: Examples of three supervision methods. (a) Binary scalar labels for global supervision. For example, 1 represents bona fide and 0 represents attack [29], [30]. (b) Pixel-wise binary mask for local supervision such as used in [21], [17]. A zero map is usually treated as ground truth of attack and a one map is for bona fide. (c) Our proposed partial attack label for AM2 attacks. The real face mask placed on an attack face is segmented and annotated as bona fide, while the rest of the attack is labeld as an attack.
  1. Regional Weighted Inference (RW) for Testing: Rather than simply averaging the final $14 \times 14$ output feature map to get a decision score, the authors apply a hand-crafted spatial weight map via a Hadamard product. Based on prior findings that the eyes are crucial for face tasks, the eye/eyebrow region is given a high weight (0.6), the mask region a very low weight (0.1), and other regions receive 0.3.
Fig. 2: An overview of our proposed PAL-RW method. The input of the model is a face image of 224×224×3 pixels, and the model is supervised by a feature map with the size of 14×14 in addition to a binary output. In the training phase, the pixel-wise label for partial attack (AM2) is produced based on 68 facial landmarks. The pixel-wise label of the real face mask in AM2 data is set to 1 (bona fide) instead of a zero map for this attack. In the inference phase, the final PAD decision score is the mean value of the regional weighted feature map for further performance improvement. The lighter the color in the region weight map, the higher the weight value, that is, the eye region contributes more to PAD decision.

The authors validate this approach using two distinct backbone networks: DeepPixBis (a standard DenseNet-based PAD model) and MixFaceNet (an ultra-efficient architecture designed for mobile face recognition).

Datasets

The method was evaluated on the Collaborative Real Mask Attack (CRMA) database. This dataset specifically addresses the masked-face PAD problem by categorizing samples into five distinct types:

  • BM0 / BM1: Unmasked and masked bona fide (real) faces.
  • AM0: Unmasked print/replay attacks.
  • AM1: Masked print/replay attacks (where the mask itself is part of the print/video).
  • AM2: Partially masked attacks (where a real physical mask is overlaid on an unmasked print/replay attack).

Key Insights and Ideas

  • Label Integrity Matters: Forcing a neural network to classify a real physical object (the mask) as a spoof hurts generalization. Treating the mask as bona fide even during an attack sequence yields cleaner gradients.
  • The Eyes Have It: When the lower half of the face is occluded, the eye and eyebrow regions contain the vast majority of discriminative clues (e.g., micro-textures, reflections) needed to distinguish a real human from a print or screen.
  • Plug-and-Play Flexibility: Because PAL-RW modifies only the ground-truth generation (training) and the final feature map pooling (inference), it requires zero modifications to the internal layers of a network, making it highly adaptable to future backbones.

Key Results

The authors conducted a step-wise ablation study to demonstrate the individual and joint benefits of PAL and RW. Performance was measured using the Average Classification Error Rate (ACER):

  • DeepPixBis Baseline: Starting at a high 29.47% ACER, adding just the RW module dropped the error to 18.58%. Combining both PAL and RW brought the ACER down to 14.81%.
  • MixFaceNet Baseline: MixFaceNet proved to be a stronger backbone, starting at 13.80% ACER. Applying the full PAL-RW method achieved the best overall performance with an ACER of 12.00% (under the unmasked threshold protocol) and 8.51% under the full dataset development threshold.
  • Partial Attack (AM2) Improvement: Under the strict evaluation protocol, the error rate specifically on partial print attacks (AM2) dropped significantly, proving that fine-grained labeling successfully guides the model to ignore the real mask and spot the fake skin.

Limitations and Future Work

While effective, the current implementation has a few limitations:

  • Rough Segmentation: The mask area is roughly estimated using 68 standard facial landmarks, which might not perfectly map to different mask shapes or slippages. More precise, automated segmentation could improve results.
  • Static Weighting: The regional weights (0.6, 0.1, 0.3) were manually assigned based on heuristic observations. The authors note that future work should explore dynamic, attention-based regional weighting that adapts automatically to different attack types and occlusions.

Summary updated

Abstract

Wearing a mask has proven to be one of the most effective ways to prevent the transmission of SARS-CoV-2 coronavirus. However, wearing a mask poses challenges for different face recognition tasks and raises concerns about the performance of masked face presentation detection (PAD). The main issues facing the mask face PAD are the wrongly classified bona fide masked faces and the wrongly classified partial attacks (covered by real masks). This work addresses these issues by proposing a method that considers partial attack labels to supervise the PAD model training, as well as regional weighted inference to further improve the PAD performance by varying the focus on different facial areas. Our proposed method is not directly linked to specific network architecture and thus can be directly incorporated into any common or custom-designed network. In our work, two neural networks (DeepPixBis and MixFaceNet) are selected as backbones. The experiments are demonstrated on the collaborative real mask attack (CRMA) database. Our proposed method outperforms established PAD methods in the CRMA database by reducing the mentioned shortcomings when facing masked faces. Moreover, we present a detailed step-wise ablation study pointing out the individual and joint benefits of the proposed concepts on the overall PAD performance.
BibTeX
@misc{fang2021partial,
  author = {Meiling Fang and Fadi Boutros and Arjan Kuijper and Naser Damer},
  title = {{Partial Attack Supervision and Regional Weighted Inference for Masked Face Presentation Attack Detection}},
  year = {2021},
  eprint = {2111.04336},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.04336},
}
Figure