Liveness Detection for Face Recognition

Gang Pan, Zhaohui Wu, Su LinInTech 2008 · DOI · OpenAlex

Abstract

Photo attacks represent the most accessible spoofing method for face recognition systems, as facial images are often publicly available. Imposters can manipulate photographs to mimic live movements, making detection a significant challenge. Video spoofing poses an even greater threat, as it incorporates physiological clues like blinking and head movement. This paper explores liveness detection techniques, focusing on spontaneous eye blinking as a non-intrusive physiological clue. We propose a robust software-based solution using Conditional Random Fields (CRF) and a novel 'eye closity' metric to distinguish live faces from static photographs using standard webcams.

Introduction: The Challenge of Face Spoofing

Face recognition systems are increasingly ubiquitous, yet they remain vulnerable to simple "low-tech" attacks. The most prevalent is the photo attack, where an imposter presents a high-quality photograph of a legitimate user. While humans intuitively recognize physiological clues like facial expressions and eye movement, translating this capability to computer vision is complex, especially in unconstrained environments.

Previous solutions have relied on depth estimation or non-rigid deformation detection. However, depth estimation is sensitive to noise and lighting, while motion detection often requires high-quality input or active user collaboration (e.g., head rotation or speaking). The goal of this research is a "software-only" solution that utilizes standard webcams without requiring extra effort from the user.

The proposed system focuses on spontaneous eye blinking, a natural physiological process. The technical approach rests on two pillars:

1. Eye Closity

Instead of processing raw pixels, the authors introduce "eye closity," a discriminative measure that maps an eye image to a single scalar value. Using an ensemble of weak classifiers, the system determines if an eye is open (negative score) or closed (positive score). This abstraction reduces computational load while preserving essential temporal information.

An example of an open eye and its corresponding negative 'closity' score.

2. Conditional Random Fields (CRF)

Blinking is a dynamic temporal sequence: Open → Ambiguous → Closed → Ambiguous → Open. Traditional models like Hidden Markov Models (HMMs) assume independent observations, which fails to capture the context of video frames. By employing a Conditional Random Field (CRF), the system incorporates a temporal window ($W$) of surrounding frames. This contextual awareness makes the model robust against transient noise, such as reflections on eyewear.

Graphical models for liveness detection: (a) Hidden Markov Model and (b) the proposed Conditional Random Field with temporal context.

Datasets and Validation

To validate the approach, two datasets were developed:

  • ZJU Eyeblink Database: 80 video clips of 20 individuals under various conditions, including different eyewear and camera angles.
  • ZJU Photo-Imposter Database: 100 spoofing attempts involving static photos, 3D movement, rotation, and bending to simulate facial depth.

Key Insights

  • Temporal Context: Accuracy improved significantly with a window size of $W=3$ (7 frames total). This aligns with the typical 250ms duration of a human blink.
  • State Modeling: Moving beyond binary "open/closed" states to include an "ambiguous" state better captures the fluid motion of eyelids.
  • Real-time Performance: The system achieves 20 frames per second (fps), making it suitable for live authentication on standard hardware.

Results

The CRF-based approach demonstrated superior performance over traditional methods:

  • Detection Rate: Achieved a 95.7% two-eye detection rate.
  • Comparative Advantage: In one-eye tests, the CRF (88.8%) significantly outperformed Cascaded AdaBoost (64.0%) and HMM (49.6%).
  • Spoofing Resistance: Only 1 out of 100 photo-spoofing clips successfully fooled the $W=3$ model.

Limitations and Future Outlook

While highly effective against photo attacks, the system faces challenges with strong eyeglass reflections that obscure the eye. Furthermore, video attacks—where an imposter plays a recording of a valid user—remain a threat, as they contain genuine blinking patterns. Future work will need to address these sophisticated spoofing methods to ensure comprehensive security.

Summary updated

Abstract

Photo attack is the cheapest and easiest spoofing approach, since one's facial image is usually very easily available for the public, for example, downloaded from the web, captured unknowingly by a camera. The imposter can rotate, shift and bend the photo before the camera like a live person to fool the authentication system. It is still a challenging task to detect whether an input face image is from a live person or from a photograph. Video spoofing is another big threat to face recognition systems, because it is very similar to live face and can be shot in front of legal user's face by a needle camera. It has many physiological clues that photo does not have, such as head movement, facial expression, blinking et al. 3D model has 3D information of face, however, it is rigid and lack of physiological information. It is also not very easy to be realistic with live person who the 3D model imitates. So photo and video are most common spoofing ways to attack face recognition system. In general, human is able to distinguish a live face and a photograph without any effort, since human can very easily recognize many physiological clues of liveness, for example, facial expression variation, mouth movement, head rotation, eye change. However, the tasks of computing these clues are often complicated for computer, even impossible for some clues under the unconstrained environment. From the static view, an essential difference between a live face and a photograph is that a live face is a fully three dimensional object while a photograph could be considered as a two dimensional planar structure. With this natural trait, Choudhary et al employed the structure from motion yielding the depth information of the face to detect live person or still photo (Choudhary et al., 1999). The disadvantages of depth information are that, firstly it is hard to estimate depth information when head is still. Secondly, the estimate is very sensitive to noise and lighting condition, becoming unreliable. Compared with photographs, another prominent characteristic of live faces is the occurrence of the non-rigid deformation and appearance change, such as mouth motion, expression variation. The accurate and reliable detection of these changes usually needs either the input data of high-quality or user collaboration. Kollreider et al applies the optical flow to the input video to obtain the information of face motion for liveness judgement Some researchers use the multi-modal approaches of face-voice against spoofing This kind of method needs voice recorder and user collaboration. An interactive approach is tried by Frischholz et al, requiring user to act an obvious response of head movement Besides, Li et al presented Fourier spectra to classify live faces or faked images, based on the assumption that the high frequency components of the photo is less than those of live face images With thermal infrared imaging camera, face thermogram also could be applied in to liveness detection Table summaries these anti-spoofing clues, in terms of data quality, hardware and user collaboration, for comparison. www.intechopen.com Liveness Detection for Face Recognition 111 Clues Data Quality Additional Hardware User Collaboration Facial expression High No Middle Depth information High No Low Mouth movement Middle No Middle Head movement High No Middle Eye blinking Low No Low
BibTeX
@inproceedings{pan2008liveness,
  author = {Gang Pan and Zhaohui Wu and Su Lin},
  title = {{Liveness Detection for Face Recognition}},
  booktitle = {InTech},
  year = {2008},
  doi = {10.5772/6397},
}
Figure