Learning Knowledge-based Prompts for Robust 3D Mask Presentation Attack Detection

Fangling Jiang, Qi Li, Bing Liu, Weining Wang, Caifeng Shan, Zhenan Sun, Ming-Hsuan Yang

Introduction

Face recognition systems have become a cornerstone of modern security and authentication. However, their widespread adoption has been met with increasingly sophisticated "presentation attacks," most notably the use of high-fidelity 3D masks. These masks, crafted from materials like silicone, resin, or plaster, can mimic the geometric and textural properties of a human face with startling accuracy, making them difficult to detect using standard RGB sensors.

Traditional countermeasures often rely on expensive multi-modal hardware (e.g., infrared or depth cameras) or temporal analysis like remote photoplethysmography (rPPG), which detects heartbeats through skin color variations. While effective in controlled settings, multi-modal sensors increase hardware costs, and rPPG is notoriously sensitive to lighting fluctuations and head movements. This paper introduces a novel framework that leverages pre-trained Vision-Language Models (VLMs) to achieve robust, single-frame 3D mask detection without specialized hardware.

The Challenge: Generalization and Spurious Correlations

Vision-Language Models like CLIP have demonstrated remarkable zero-shot generalization across various computer vision tasks. However, applying them to 3D mask presentation attack detection (PAD) presents two primary hurdles:

  1. Prompt Granularity: Generic prompts such as "a photo of a 3D mask" lack the descriptive power to capture the micro-textures and material properties (e.g., rigid edges, synthetic pores) that distinguish a mask from real skin.
  2. Spurious Correlations: Deep learning models often achieve high accuracy by "cheating"—learning to associate specific backgrounds or lighting conditions with a label rather than focusing on the face itself. This leads to catastrophic failure when the model is tested in a new environment (cross-dataset scenarios).

Proposed Methodology

To address these issues, the authors propose a Knowledge-based Prompt Learning Framework characterized by two core innovations: Knowledge Graph-driven prompting and Causal Learning for feature purification.

1. Knowledge Graph Prompts with Visual Filtering

Instead of relying on manually engineered or generic prompts, the framework utilizes a specialized Knowledge Graph (MaskPAD-KG) containing expert domain knowledge about mask materials and facial structures. By querying entities like "silicone," "resin," and "rigid shape," and using GPT-4 to generate detailed descriptions, the model gains access to a sophisticated vocabulary of spoofing cues.

To ensure these prompts are relevant to the specific image being analyzed, the authors introduce a Visual-Specific Knowledge Filter. This attention-based mechanism dynamically weights the knowledge entities based on the visual input. For instance, if the input image contains a resin mask, the filter prioritizes resin-related textual descriptions, allowing for a more precise alignment between the visual and textual modalities.

2. Eliminating Spurious Correlations via Causal Learning

To ensure the model learns "causal" features (the actual characteristics of the mask) rather than "non-causal" features (the background), the authors designed a training paradigm rooted in causal graph theory. The system decomposes the image into local patches and identifies those that do not correlate with the ground-truth label—typically background elements like walls or furniture. By mathematically maximizing the entropy of the similarity distribution for these irrelevant patches, the model is forced to ignore them, focusing its representational power on the face and mask boundaries.

Experimental Results

The framework was evaluated against several benchmark datasets, including 3DMAD, HKBU-MARs V1+, and HiFiMask, with a specific focus on cross-dataset generalization.

Performance and Efficiency

  • State-of-the-Art Accuracy: In cross-dataset evaluations, the proposed method achieved a significant reduction in the Half Total Error Rate (HTER), showing improvements ranging from 44% to 94% over the previous leading method, FASTEN.
  • Inference Speed: Unlike multi-frame methods that require processing optical flow or temporal sequences, this model operates on a single frame. This results in an inference time of just 3.91 ms, nearly six times faster than temporal-based competitors.
  • Interpretability: Visualizations of the model’s attention maps confirm that it focuses on 3D-rich structures (nose, eyes) for real faces and critical junction edges or surface textures for 3D masks.

Conclusion and Future Work

By combining the vast general knowledge of VLMs with expert-level domain knowledge and causal learning, this research provides a robust and efficient solution for 3D mask detection. The method's ability to generalize across different cameras, lighting conditions, and mask materials marks a significant step forward for secure face recognition.

Currently, the MaskPAD-KG is constructed manually, which may introduce expert bias and limit scalability. Future research will focus on automating the construction of more comprehensive and diverse knowledge graphs to further enhance the model's ability to detect emerging spoofing threats.

Summary updated

Abstract

3D mask presentation attack detection is crucial for protecting face recognition systems against the rising threat of 3D mask attacks. While most existing methods utilize multimodal features or remote photoplethysmography (rPPG) signals to distinguish between real faces and 3D masks, they face significant challenges, such as the high costs associated with multimodal sensors and limited generalization ability. Detection-related text descriptions offer concise, universal information and are cost-effective to obtain. However, the potential of vision-language multimodal features for 3D mask presentation attack detection remains unexplored. In this paper, we propose a novel knowledge-based prompt learning framework to explore the strong generalization capability of vision-language models for 3D mask presentation attack detection. Specifically, our approach incorporates entities and triples from knowledge graphs into the prompt learning process, generating fine-grained, task-specific explicit prompts that effectively harness the knowledge embedded in pre-trained vision-language models. Furthermore, considering different input images may emphasize distinct knowledge graph elements, we introduce a visual-specific knowledge filter based on an attention mechanism to refine relevant elements according to the visual context. Additionally, we leverage causal graph theory insights into the prompt learning process to further enhance the generalization ability of our method. During training, a spurious correlation elimination paradigm is employed, which removes category-irrelevant local image patches using guidance from knowledge-based text features, fostering the learning of generalized causal prompts that align with category-relevant local patches. Experimental results demonstrate that the proposed method achieves state-of-the-art intra- and cross-scenario detection performance on benchmark datasets.
BibTeX
@article{jiang2025learning,
  author = {Fangling Jiang and Qi Li and Bing Liu and Weining Wang and Caifeng Shan and Zhenan Sun and Ming-Hsuan Yang},
  title = {{Learning Knowledge-based Prompts for Robust 3D Mask Presentation Attack Detection}},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  doi = {10.1109/tpami.2025.3618630},
}
Figure