TL;DR
The widespread use of face masks has severely degraded the accuracy of face Presentation Attack Detection (PAD) systems, particularly when dealing with "partial attacks" where a real mask is placed over a spoofed face. This paper introduces PAL-RW, a two-pronged solution that uses fine-grained Partial Attack Labeling (PAL) during training to correctly identify real mask regions as bona fide, combined with Regional Weighted Inference (RW) during testing to focus on the highly discriminative eye region. This architecture-agnostic approach significantly reduces error rates across multiple neural network backbones.
What Problem Does This Paper Address?
Face recognition and anti-spoofing systems (PAD) faced an unexpected challenge with the COVID-19 pandemic: widespread face mask adoption. Standard PAD algorithms often misclassify real, masked faces as presentation attacks (spoofs), or fail to catch sophisticated "partial attacks."
A partial attack occurs when an attacker places a physically real face mask over a printed photo or a digital screen displaying a fake face. Traditional pixel-wise supervision methods treat the entire attack image as a "zero map" (100% spoof). However, because the mask itself is a real object, forcing the network to learn that a real mask is an "attack" introduces significant noise during training. Furthermore, standard evaluation methods treat all facial regions equally during inference, ignoring the fact that when the lower face is covered, the eye region becomes the most critical area for verifying authenticity.
Methods
The authors propose PAL-RW, which consists of two main components that can be integrated into any pixel-wise PAD network architecture:
- Partial Attack Label (PAL) for Training: Instead of labeling the entire partial attack image as a spoof, the authors use 68 facial landmarks to segment the boundary of the face mask. Pixels corresponding to the real face mask are labeled as
1(bona fide), while the uncovered, spoofed regions of the face are labeled as0(attack). This fine-grained labeling provides cleaner supervision and helps the model converge effectively.
- Regional Weighted Inference (RW) for Testing: Rather than simply averaging the final $14 \times 14$ output feature map to get a decision score, the authors apply a hand-crafted spatial weight map via a Hadamard product. Based on prior findings that the eyes are crucial for face tasks, the eye/eyebrow region is given a high weight (
0.6), the mask region a very low weight (0.1), and other regions receive0.3.
The authors validate this approach using two distinct backbone networks: DeepPixBis (a standard DenseNet-based PAD model) and MixFaceNet (an ultra-efficient architecture designed for mobile face recognition).
Datasets
The method was evaluated on the Collaborative Real Mask Attack (CRMA) database. This dataset specifically addresses the masked-face PAD problem by categorizing samples into five distinct types:
- BM0 / BM1: Unmasked and masked bona fide (real) faces.
- AM0: Unmasked print/replay attacks.
- AM1: Masked print/replay attacks (where the mask itself is part of the print/video).
- AM2: Partially masked attacks (where a real physical mask is overlaid on an unmasked print/replay attack).
Key Insights and Ideas
- Label Integrity Matters: Forcing a neural network to classify a real physical object (the mask) as a spoof hurts generalization. Treating the mask as bona fide even during an attack sequence yields cleaner gradients.
- The Eyes Have It: When the lower half of the face is occluded, the eye and eyebrow regions contain the vast majority of discriminative clues (e.g., micro-textures, reflections) needed to distinguish a real human from a print or screen.
- Plug-and-Play Flexibility: Because PAL-RW modifies only the ground-truth generation (training) and the final feature map pooling (inference), it requires zero modifications to the internal layers of a network, making it highly adaptable to future backbones.
Key Results
The authors conducted a step-wise ablation study to demonstrate the individual and joint benefits of PAL and RW. Performance was measured using the Average Classification Error Rate (ACER):
- DeepPixBis Baseline: Starting at a high 29.47% ACER, adding just the RW module dropped the error to 18.58%. Combining both PAL and RW brought the ACER down to 14.81%.
- MixFaceNet Baseline: MixFaceNet proved to be a stronger backbone, starting at 13.80% ACER. Applying the full PAL-RW method achieved the best overall performance with an ACER of 12.00% (under the unmasked threshold protocol) and 8.51% under the full dataset development threshold.
- Partial Attack (AM2) Improvement: Under the strict evaluation protocol, the error rate specifically on partial print attacks (AM2) dropped significantly, proving that fine-grained labeling successfully guides the model to ignore the real mask and spot the fake skin.
Limitations and Future Work
While effective, the current implementation has a few limitations:
- Rough Segmentation: The mask area is roughly estimated using 68 standard facial landmarks, which might not perfectly map to different mask shapes or slippages. More precise, automated segmentation could improve results.
- Static Weighting: The regional weights (0.6, 0.1, 0.3) were manually assigned based on heuristic observations. The authors note that future work should explore dynamic, attention-based regional weighting that adapts automatically to different attack types and occlusions.
Summary updated