Spoofing 2D Face Detection: Machines See People Who Aren't There

Michael McCoyd, David A. WagnerarXiv 2016 · OpenReview

Abstract

Machine learning is increasingly used to make sense of the physical world yet may suffer from adversarial manipulation. We examine the Viola-Jones 2D face detection algorithm to study whether images can be created that humans do not notice as faces yet the algorithm detects as faces. We show that it is possible to construct images that Viola-Jones recognizes as containing faces yet no human would consider a face. Moreover, we show that it is possible to construct images that fool facial detection even when they are printed and then photographed.

Introduction

As computer vision systems are deployed in security-critical "attended settings"—such as checkpoints where a human guard monitors automated systems—the nature of adversarial threats shifts. While most research focuses on facial recognition (identity), this study investigates facial detection (presence). The goal is to create "ghost faces": images that appear as harmless textures or noise to a human observer but are registered as valid human faces by a machine. Such spoofs could trigger false alarms or bypass occupancy-based security measures without raising human suspicion.

The Viola-Jones Mechanism

The researchers targeted the Viola-Jones (VJ) algorithm, a long-standing industry standard for real-time face detection. VJ operates by identifying "Haar-like" features—mathematical representations of the differences in average intensity between adjacent rectangular regions of an image. For example, the bridge of the nose is typically lighter than the eyes, and the forehead is lighter than the eye sockets.

Viola-Jones relies on specific rectangular Haar-like features to detect faces.

The attack succeeds by preserving these specific mathematical relationships while destroying the global semantic features (like eyes, nose, and mouth) that the human brain requires to recognize a face.

Methodology: Creating Stealthy Ghost Faces

Because VJ provides only a binary output (face/no-face), the authors developed a feedback-guided stochastic search called Random Shift. Starting with a real face, the algorithm iteratively moves pixels toward a "cover image" (e.g., a granite texture). A change is only accepted if the VJ "oracle" still detects a face. This process effectively "walks" the image along the classifier's decision boundary until it is unrecognizable to humans.

To ensure these attacks work in the real world, the authors modeled an "Analog Channel". This simulation accounts for physical distortions like upscaling, Gaussian blur, and sensor noise. By requiring an image to pass detection in multiple simulated trials, they generated robust adversarial images capable of surviving the transition from digital display to camera capture.

Human vs. Machine Perception

A key challenge in attended settings is remaining "stealthy." Humans are remarkably adept at spotting faces even in heavy noise. The researchers found a specific threshold where the machine continues to detect a face while the human eye fails to see anything but the cover texture.

At high blend percentages, humans fail to perceive the face that the machine still detects.

Key Results and Implications

The study successfully generated images that appeared as white noise or granite to humans but were detected as faces by VJ. In physical testing, a robust "abstract dot" image was detected as a face 97% of the time when displayed on a tablet and captured by a webcam.

While modern systems have moved toward Deep Neural Networks, this research highlights a fundamental vulnerability: the gap between mathematical feature detection and semantic human understanding. Future work must address whether these "ghost faces" can be tuned to spoof specific identities in facial recognition systems.

Summary updated

Abstract

Machine learning is increasingly used to make sense of the physical world yet may suffer from adversarial manipulation. We examine the Viola-Jones 2D face detection algorithm to study whether images can be created that humans do not notice as faces yet the algorithm detects as faces. We show that it is possible to construct images that Viola-Jones recognizes as containing faces yet no human would consider a face. Moreover, we show that it is possible to construct images that fool facial detection even when they are printed and then photographed.
BibTeX
@misc{mccoyd2016spoofing,
  author = {Michael McCoyd and David A. Wagner},
  title = {{Spoofing 2D Face Detection: Machines See People Who Aren't There}},
  year = {2016},
  eprint = {1608.02128},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1608.02128},
}
Figure