Primitive-Driven Compositional Forensic Visual Prompting for Open-World Face Anti-Spoofing

Fangling Jiang, Qi Li, Bing Liu, Weining Wang, Quilin Huang, Zhenan Sun, Ming-Hsuan YangarXiv 2026 · arXiv

Here is a concise, engaging summary of the paper.

TL;DR

This paper introduces a purely visual, compositional prompt learning framework for open-world face anti-spoofing that bypasses text encoders entirely. By decomposing complex, unseen spoofing attacks into a shared library of learnable "micro-forensic primitives" and dynamically combining them based on global context, the model achieves state-of-the-art generalization across diverse datasets and novel attack types.

What Problem Does This Paper Address?

Face anti-spoofing (FAS) systems protect facial recognition systems from presentation attacks like photos, video replays, or 3D masks. However, real-world deployment is an "open-world" challenge characterized by two distinct hurdles:

  1. Covariate Shift: Variations in lighting, cameras, and backgrounds between the training and deployment environments.
  2. Semantic Shift: The continuous emergence of entirely new, unseen attack types (e.g., silicone masks, advanced makeup, or partial face covers) that were completely absent from the training data.

Recently, researchers have used large vision-language models (like CLIP) to improve generalization via text prompts. However, the authors argue that text-driven approaches are fundamentally limited for forensics. Text encoders favor high-level semantic abstractions, which inadvertently suppress the low-level, high-frequency visual details (like subtle texture degradation or edge anomalies) critical for spotting a spoof. Furthermore, complex physical artifacts are incredibly difficult to describe precisely in natural language.

Methods

Instead of relying on rigid text descriptions, the authors propose a framework based on a simple hypothesis: unseen attacks are often just novel combinations of recurring visual cues. For instance, a silicone mask attack might combine geometric rigidity, abnormal reflections, and altered skin texture—all cues that appear individually in other seen attacks.

The proposed framework, built on a frozen Vision Transformer (ViT) backbone from CLIP, operates entirely in the visual feature space through three main steps:

  1. Patch-Aware Primitive Refinement: The model initializes a set of learnable vectors called micro-forensic primitives. Using a patch-aware attention mechanism, these primitives act as queries against local image patches to extract and specialize in fine-grained, reusable forensic evidence units.
  2. Globally Guided Primitive Routing: Class-specific global contextual prompts interact with the image tokens to capture holistic physical characteristics of the input. A lightweight routing network uses these global tokens to dynamically compute weights over the refined primitives.
  3. Compositional Prompting: The primitives are adaptively combined based on the routing weights to form input-specific compositional forensic prompts used for final real/spoof classification.
Fig. 2. Overview of the proposed compositional forensic diagnosis framework for open-world face anti-spoofing. Built on a frozen ViT-based vision foundation model, the framework learns hierarchical purely visual prompts without language encoding. At each stage, micro-forensic primitives are first refined through patch-aware interactions to summarize reusable and composable fine-grained forensic units. These forensic units are then dynamically composed into class- specific primitive evidence prompts according to global priors and augmented with global contextual prompts to generate stage-wise compositional forensic visual prompts. Multi-stage prompts are finally aggregated for real/spoof discrimination. By representing heterogeneous attacks as dynamic compositions of reusable forensic evidence, the framework enables unified reasoning over diverse unseen attack types.

Datasets

The framework was evaluated across nine cross-dataset open-world protocols using eight public benchmarks:

  • Source Domains (Seen Attacks): CASIA-MFSD, Replay-Attack, MSU-MFSD, and OULU-NPU (primarily consisting of conventional print and replay attacks).
  • Target Domains (Unseen Attacks): HQ-WMCA (covering visible, depth, and thermal modalities with attacks like rigid masks, mannequins, and makeup) and SiW-Mv2 (featuring 14 diverse attack types).
  • Large-Scale Evaluation: CASIA-SURF and CASIA-SURF CeFA (introducing cross-ethnicity and multi-modal variations).

Key Insights and Ideas

  • Visual Prompts Capture High Frequencies Better: Fourier analysis reveals that the proposed purely visual prompts retain significantly more mid- and high-frequency energy than text-guided prompt baselines (like CoOp). This allows them to lock onto fine-grained physical anomalies that text descriptions miss.
  • Emergent Specialization: Primitives are not assigned predefined semantic meanings during initialization. Instead, through joint optimization across diverse categories, individual primitives naturally learn to specialize in distinct, reusable forensic cues (e.g., skin textures, boundary transitions, or reflections).
  • Hierarchical Evolution: Layer-wise visualizations show that the prompts progressively guide the frozen foundation model from broad, global facial structures in shallow layers down to highly localized spoof traces (like the edges of paper glasses or mouth cutouts) in deeper layers.
Visualization of attention regions shifting from global context to localized spoofed regions across layers

Key Results

The proposed method delivers state-of-the-art performance across all evaluated open-world protocols:

  • Substantial Error Reduction: On the challenging CASIA-MFSD to SiW-Mv2 ($C \rightarrow W$) protocol, it achieves a Half Total Error Rate (HTER) of 10.14% and an AUC of 95.26%, outperforming the previous best prompt-based method (MEFAS, which scored 13.65% HTER).
  • Cross-Protocol Consistency: Across all four source-to-SiW-Mv2 protocols, it achieves an average HTER of 22.26%, representing a 29.51% relative reduction in error over the second-best method.
  • Robust Localization: Qualitative attention maps demonstrate that the model accurately localizes attack-specific regions across highly heterogeneous spoof types, such as partial masks or cosmetic alterations.

Limitations and Future Work

Despite its strong performance, the model struggles with highly subtle attacks that preserve natural facial appearance. Specifically, makeup obfuscation and cosmetic makeup yielded the highest error rates (63.64% and 37.25%, respectively), as they introduce minimal geometric or textural anomalies.

For future work, the authors plan to explore more explicit semantic interpretations of the learned primitives to improve explainability and extend the framework to multimodal face anti-spoofing (e.g., integrating infrared and depth data).

Summary updated

Abstract

Open-world face anti-spoofing must address both covariate and semantic shifts: source and target domains differ in imaging conditions, while target domains contain diverse attack types absent from training. Existing prompt-based approaches often express spoofing through category semantics or language guidance, which is effective for modeling high-level concepts but is less suited to explicitly capturing the evolving fine-grained and spatially heterogeneous forensic evidence of unseen attacks. Motivated by the hypothesis that many unseen attacks can be characterized by new combinations of recurring visual cues, we propose a compositional forensic visual prompt learning framework that operates entirely in the visual feature space. Built on a frozen ViT-based vision foundation model, the framework employs patch-aware attention to refine a shared set of learnable micro-forensic primitives into localized forensic evidence units derived from image patches. Class-specific global contextual prompts then provide input-dependent routing weights that adaptively select and compose these primitives into compositional forensic visual prompts for real/spoof discrimination. The primitives are not assigned predefined semantic meanings; instead, their specialization and reuse emerge from shared parameterization and joint optimization across categories. Extensive experiments on nine open-world protocols demonstrate state-of-the-art performance, strong cross-domain generalization, and robust adaptation to unseen attacks.
BibTeX
@misc{jiang2026primitive,
  author = {Fangling Jiang and Qi Li and Bing Liu and Weining Wang and Quilin Huang and Zhenan Sun and Ming-Hsuan Yang},
  title = {{Primitive-Driven Compositional Forensic Visual Prompting for Open-World Face Anti-Spoofing}},
  year = {2026},
  eprint = {2608.17351},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.17351},
}
Figure