Tomatoes, Potatoes, and Onions: Questioning the Need for Faces in Face Presentation Attack Detection

Guray Ozgur, Fadi Boutros, Naser DamerarXiv 2026 · arXiv

TL;DR

Does face anti-spoofing actually require faces? This paper demonstrates that a presentation attack detection (PAD) model trained entirely on videos of tomatoes, potatoes, and onions can successfully detect spoofing attacks on human faces, achieving an impressive 92.70% average AUC on standard benchmarks. By shifting the focus from facial semantics to recapture artifacts, this work opens up new pathways for privacy-preserving, identity-independent biometric security.

What Problem Does This Paper Address?

Face Presentation Attack Detection (PAD)—often called face anti-spoofing—is the critical line of defense protecting facial recognition systems from being fooled by printed photos or replayed videos. Traditionally, the computer vision community has treated PAD as a face-specific problem. This framing has driven the collection of ever-larger datasets of real human faces, raising severe privacy, consent, and demographic bias concerns.

The authors challenge this fundamental premise. They argue that the visual artifacts introduced by a spoofing attack—such as the moiré patterns of a screen, the halftoning of a printout, specular highlights, and recapture noise—are properties of the presentation instrument and capture pipeline, not the object being captured. If the face itself is incidental, can we train a reliable spoof detector without using human faces at all?

Methods

To test their hypothesis, the authors adopt a foundation-model-based PAD architecture based on FoundPAD. The pipeline uses a frozen CLIP ViT-B/16 image encoder adapted with lightweight, parameter-efficient tuning via Low-Rank Adaptation (LoRA) on the query and value projections of the attention blocks. A simple linear two-class head sits on top to classify presentations as either bona fide (real) or attack (spoof).

Crucially, the pipeline applies no face detectors, facial cropping, or landmark alignment. The authors also introduce a normalization correction to align the data pipeline with CLIP's native pretraining statistics, resolving a common preprocessing mismatch found in prior literature.

Datasets

The core contribution of the data strategy is the introduction of TPO (Tomatoes, Potatoes, and Onions), a controlled, face-free PAD dataset.

  • TPO Dataset: Contains 12,480 total presentations across 78 distinct vegetable identities (26 per species), chosen for their varying surface textures and reflectiveness. It features balanced numbers of bona fide captures, print attacks (high-resolution printouts recaptured on video), and replay attacks (videos replayed on smartphones/tablets and recaptured).
Figure 1: Same presentation process, different objects. TPO examples (top) and Idiap Replay-Attack examples (bottom) contain the same PAD structure: bona fide camera observations and print or display attacks recaptured by a second camera. Replacing faces with tomatoes, potatoes, and onions preserves the acquisition and presentation-instrument cues while removing facial identity, which makes the cross-object transfer test possible. The TPO dataset is released at https://github.com/gurayozgur/TPO.

For cross-dataset evaluation on human faces, the authors benchmark their models on four standard face PAD datasets:

  • MSU-MFSD (M)
  • CASIA-FASD (C)
  • Idiap Replay-Attack (I)
  • OULU-NPU (O)
  • SynthASpoof: A state-of-the-art synthetic-face dataset used as a privacy-preserving control baseline.

Key Insights and Ideas

  • Faces are Incidental: The primary cues for cross-dataset generalization are recapture artifacts. Because these artifacts map to the hardware and environment rather than the subject, any textured object can serve as a proxy for training.
  • Artifact Reality Beats Facial Semantics: Training on real vegetables significantly outperforms training on photorealistic synthetic human faces (SynthASpoof achieves 81.02% AUC vs. TPO's 92.70%). Preserving realistic physical recapture artifacts is far more important than preserving facial structure.
  • Complementary Data Cues: When TPO data replaces a portion of real-face data under a fixed optimization budget, cross-dataset performance improves. This indicates that face-free data provides complementary, non-redundant information about presentation artifacts.
  • Beyond Simple Frequency Shortcuts: Frequency analysis shows that the model isn't just picking up on a single global spectral trick (like high-frequency noise). The learned representations capture a richer, multi-faceted combination of spatial and textural presentation cues.
Figure 3: The transferable cue is not one global frequency signature. Radial log-magnitude spectra for M, C, I, O, TPO, and SynthASpoof. Bona fide curves in (a) differ strongly by dataset and crop. Attack - bona fide residuals in (b) also disagree in direction: TPO gains high-frequency energy, C, I, O, and SynthASpoof lose it, and M remains near zero. Curves average 400 images per class and dataset; shading gives 95% confidence intervals. Colors match Fig. 2. Images are grayscale, resized to 256×256, standardized per image, and Hann-windowed before the FFT.

Key Results

  • Zero-Face Training Success: The FoundPAD model trained exclusively on TPO vegetables achieved an average video-level Area Under the ROC Curve (AUC) of 92.70% and a Half Total Error Rate (HTER) of 14.15% across the four human face benchmarks (MCIO).
  • Boosting Multi-Source Regimes: Integrating TPO into multi-source face training pipelines consistently boosted generalization, raising the average cross-dataset AUC from 92.11% to 96.96% and cutting the HTER nearly in half (from 14.72% to 7.84%).
Figure 2: Training source changes the embedding geometry. FoundPAD visual embeddings for the same class-balanced samples from M, C, I, O, TPO, and SynthASpoof. Panels (a-b) use the TPO-trained encoder; panels (c-d) use the O-trained encoder. Dataset coloring in (a,c) shows stronger source-domain organization after TPO training, while PAD coloring in (b,d) shows a clearer cross-domain bona fide/attack partition after O training. Each dataset contributes 250 bona fide and 250 attack images. Each encoder is projected separately with fixed-seed PCA-t-SNE; axes are arbitrary.

Limitations and Future Work

The conclusions of this study are explicitly bounded by print and replay attacks within the visible (RGB) spectrum. The authors note that it remains an open question whether face-free training can transfer to attacks that rely heavily on 3D geometry, such as rigid three-dimensional masks, or to alternative sensing modalities like infrared (IR) and depth imaging.

Furthermore, while the downstream training data is entirely face-free, the underlying CLIP foundation model was pretrained on large-scale internet imagery that included human faces. Disentangling the exact minimal representation prior required to bootstrap face-free PAD remains an exciting direction for future research.

Summary updated

Abstract

Face presentation attack detection (PAD) is traditionally formulated as a face-specific problem, although many of the visual artifacts introduced by print, replay, and recapture processes are not inherently tied to facial appearance. In this work, we investigate whether transferable PAD representations can be learned without using faces during downstream PAD training. To this end, we introduce TPO, a controlled face-free presentation attack dataset consisting of bona fide, print, and replay recordings of, almost randomly chosen, tomatoes, potatoes, and onions acquired under protocols that closely mirror conventional face PAD datasets. Using a foundation-model-based PAD architecture, we demonstrate that a detector trained on TPO achieves an average AUC of 92.70% across four standard cross-dataset face PAD benchmarks, outperforming training on synthetic faces and remaining competitive with models trained on real face datasets. Conversely, models trained on face PAD datasets transfer consistently above chance to TPO, suggesting that the learned representations capture characteristics of the presentation process rather than object semantics. Furthermore, incorporating TPO into conventional face PAD training consistently improves cross-dataset performance under fixed optimization budgets, indicating that face-free data provides complementary information rather than simply additional training samples. Finally, representation and frequency analyses provide further evidence that transferable PAD representations cannot be explained by a single spectral artifact but instead encode richer presentation cues shared across object categories. Together, these results provide empirical evidence that transferable presentation attack representations can be learned independently of facial content, opening new opportunities for privacy-preserving and identity-independent PAD development.
BibTeX
@misc{ozgur2026tomatoes,
  author = {Guray Ozgur and Fadi Boutros and Naser Damer},
  title = {{Tomatoes, Potatoes, and Onions: Questioning the Need for Faces in Face Presentation Attack Detection}},
  year = {2026},
  eprint = {2608.21455},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.21455},
}
Figure