Fast face identification under varying pose from a single 2-D model view

Yongsheng Gao, M.k.h. Leung, W. Wang, Hui SunIEE Proceedings - Vision Image and Signal Processing 2001 · DOI

TL;DR

This paper introduces a computationally efficient method for recovering the 3D pose of a face from a single 2D image. By leveraging the inherent symmetry of the human head, it transforms a rotated face into a fronto-parallel view, enabling real-time face identification without the need for extensive storage of multiple virtual views.

What Problem Does This Paper Address?

Face recognition systems have made significant strides, but a persistent challenge is accurately identifying faces when their orientation (pose) changes. Traditional methods often rely on having multiple example views of a person's face or using complex, computationally expensive 3D models to synthesize new views. This paper tackles the problem of face identification when only a single 2D image of a person is available, and the input image might be rotated in depth. The goal is to achieve robust identification despite these pose variations, while also addressing the computational and storage overhead of existing approaches.

Methods

The core of the paper is a novel, computationally efficient 3D pose recovery method. It exploits the assumption of human head symmetry. Given a single 2D face image, the system estimates the degree of head rotation ($\theta$) by analyzing the ratio of distances from the eyes to the face's symmetry axis. This ratio is related to the head's geometry and the viewing angle.

Once the rotation angle is estimated, the system applies a transformation to convert the 2D rotated face image into a 2D fronto-parallel view. This is achieved by treating the face as a texture-mapped cylinder and performing a proper rotation followed by orthographic projection. Crucially, this method avoids generating multiple virtual views, which is a common but storage-intensive approach. Instead, it directly transforms the input into a canonical pose for subsequent identification. The paper also details the mathematical derivations for calculating the rotation angle based on the symmetry ratio and head model parameters.

For evaluation, the authors implemented three face identification systems:

  1. Modified Hausdorff Distance (MHD): Based on face edge maps.
  2. Line Segment Hausdorff Distance (LHD): Based on face line edge maps (LEM), which are derived from edge maps and offer structural information.
  3. Eigenface: A well-established technique using principal component analysis.

Each of these systems was tested both with and without the proposed 3D pose recovery method.

Datasets

The experiments were conducted using the University of Bern face database. This database contains frontal views of 30 people, with each person having multiple images captured at different head pose variations. The authors used one fronto-parallel image per person as the "single model" and tested the systems against 4 images with poses looking to the right and left for each person, totaling 120 test images. Clipfaces (the facial regions used for recognition) were generated and normalized for scale and orientation by aligning eye locations.

Key Insights and Ideas

  • Leveraging Symmetry for Efficiency: The paper demonstrates that the inherent symmetry of the human head can be a powerful cue for pose estimation, leading to a computationally much cheaper solution than full 3D model reconstruction.
  • Single View to Canonical Pose: The core idea of transforming a rotated face into a fronto-parallel view directly is key. This bypasses the need to store numerous synthesized views, significantly reducing storage requirements.
  • Real-time Applicability: By avoiding complex 3D model computations and multiple view generation, the proposed method is suitable for real-time face identification applications.
  • Edge-based Recognition Benefits: The experiments show that pose recovery significantly improves recognition rates for edge-based methods (MHD and LHD), suggesting that structural features are robust to pose changes when normalized.

Key Results

The proposed 3D pose recovery method consistently improved recognition rates across all three tested identification systems:

  • MHD: Recognition rates increased by 10% (Top1: 50% to 60%).
  • LHD: Recognition rates increased by 15.8% (Top1: 54.2% to 70%).
  • Eigenface: Recognition rates increased by 10% (Top1: 70% to 80%).

These improvements were observed for Top1, Top3, and Top5 classifications. The paper also reported that the average distances of correct matches ( $d_{cor}$ ) decreased significantly after pose recovery, indicating that the rotated faces became more similar to their fronto-parallel counterparts. For instance, the $d_{cor}$ for Eigenface decreased by 27.27%. The authors also noted that the system's performance was comparable to human performance benchmarks.

Limitations and Future Work

  • Limited Pose Variation: The current method primarily addresses rotation in depth (yaw). It cannot recover varying poses looking upwards and downwards (pitch) because these movements do not alter the symmetry about the vertical axis of the image plane.
  • Future Research: The authors suggest investigating pose recovery with respect to longitudinal pose variations (pitch) as a direction for future research. They also highlight the need for further exploration into how to automatically establish corresponding symmetric points on faces, as this is crucial for the symmetry-based approach.

Summary updated

Abstract

One of the key remaining problems in face recognition is that of handling the variability in appearance due to changes in pose. The authors present a simple and computationally efficient 3-D pose recovery methodology. It addresses the computationally expensive problem of current generic 3-D model pose recovery methods and thus is able to be applied in real-time applications. Compared with the virtual view methods, the face identification system with the proposed pose recovery method demands much less storage space as it transforms the 2-D rotated face to the 2-D fronto-parallel view for subsequent identification rather than generating multiple virtual views for a single input face. Experiments evaluating the effectiveness of the technique are reported. The systems are compared with human performances and existing techniques.
BibTeX
@inproceedings{gao2001fast,
  author = {Yongsheng Gao and M.K.H. Leung and W. Wang and Hui Sun},
  title = {{Fast face identification under varying pose from a single 2-D model view}},
  booktitle = {IEE Proceedings - Vision Image and Signal Processing},
  year = {2001},
  doi = {10.1049/ip-vis:20010377},
}
Figure