Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor

Ryan Connaughton, Kevin W. Bowyer, Patrick J. FlynnMAICS 2011 · OpenReview

Introduction

Traditional biometric systems often rely on a single modality, such as fingerprints or facial recognition. While effective, these "uni-biometric" systems are susceptible to failure when a specific trait is obscured or difficult to capture at a distance. Multi-modal fusion—combining multiple biometric traits—offers a solution, yet it faces challenges like hardware complexity and the use of "chimeric" datasets (randomly paired traits from different individuals).

Researchers at the University of Notre Dame have addressed these issues by developing a system that captures genuine paired face and iris data from a single "stand-off" video sensor. By utilizing the Iris on the Move (IOM) system, they achieved a 93.2% recognition rate, demonstrating the viability of multi-modal fusion in high-throughput, "on-the-move" environments.

Methodology

The proposed system leverages the IOM sensor, which consists of three vertically arranged near-infrared (NIR) video cameras. Although originally designed for iris recognition, the sensor's field of view is wide enough to capture the entire face, allowing for "face recognition for free" without additional hardware.

System architecture for fusing face and iris biometrics.

Preprocessing and Detection

The pipeline begins with Frame Stitching and Histogram Matching. Since the cameras capture different parts of the subject at varying heights, the system aligns the three video feeds using template matching. Histogram matching is then applied to normalize illumination differences between the cameras, creating a seamless stitched frame.

The process of stitching and histogram matching to create a unified face image.

For detection, the system employs the Viola-Jones cascade detector for faces. Eye detection uses a specialized template matching approach focused on specular highlights (NIR glints), which provides robustness in lower-resolution video frames.

Matching and Fusion

The system extracts features and calculates match scores for both modalities:

  • Face Matching: Utilizes the "Eigenface" approach with a Mahalanobis cosine metric.
  • Iris Matching: Employs a modified Daugman algorithm to calculate normalized Hamming distances.

The core contribution lies in the fusion of these scores. The researchers compared several strategies, including multi-sample (taking the best score across frames), rank-level (Borda Count), and score-level fusion. Weighted score-level fusion (SumBoth), which performs a weighted summation of normalized face and iris scores, proved to be the most effective method.

Experimental Setup

The study utilized a genuine multi-modal dataset comprising 1,886 IOM video sets from 363 unique subjects. Unlike chimeric datasets, this data captures real-world dependencies between face and iris traits. The gallery consisted of high-quality, manually screened images, while the probes were the "on-the-move" video frames captured as subjects walked through the portal.

Results

The fusion approach significantly outperformed single-modality recognition. While face recognition alone (MinFace) achieved a 62.6% rank-one recognition rate and iris recognition (MinIris) reached 87.8%, the SumBoth fusion method achieved 93.2%. This 5.4% absolute improvement over the best single modality highlights the resilience of multi-modal systems; if one trait is partially obscured, the other can still facilitate a confident match.

Discussion and Future Work

Despite the success, limitations remain. Motion blur and defocus in video frames occasionally hindered matching. Furthermore, the Viola-Jones detector struggled with partial faces when subjects were not fully within the camera's field of view. Future research will focus on incorporating "quality metrics" to automatically weight frames based on sharpness and exploring more sophisticated partial-face matching algorithms to further enhance robustness.

Summary updated

BibTeX
@inproceedings{connaughton2011fusion,
  author = {Ryan Connaughton and Kevin W. Bowyer and Patrick J. Flynn},
  title = {{Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor}},
  booktitle = {MAICS},
  year = {2011},
  url = {https://openreview.net/forum?id=GcF1kkMa4Hc},
}
Figure