Introduction
Traditional biometric systems often rely on a single modality, such as fingerprints or facial recognition. While effective, these "uni-biometric" systems are susceptible to failure when a specific trait is obscured or difficult to capture at a distance. Multi-modal fusion—combining multiple biometric traits—offers a solution, yet it faces challenges like hardware complexity and the use of "chimeric" datasets (randomly paired traits from different individuals).
Researchers at the University of Notre Dame have addressed these issues by developing a system that captures genuine paired face and iris data from a single "stand-off" video sensor. By utilizing the Iris on the Move (IOM) system, they achieved a 93.2% recognition rate, demonstrating the viability of multi-modal fusion in high-throughput, "on-the-move" environments.
Methodology
The proposed system leverages the IOM sensor, which consists of three vertically arranged near-infrared (NIR) video cameras. Although originally designed for iris recognition, the sensor's field of view is wide enough to capture the entire face, allowing for "face recognition for free" without additional hardware.
Preprocessing and Detection
The pipeline begins with Frame Stitching and Histogram Matching. Since the cameras capture different parts of the subject at varying heights, the system aligns the three video feeds using template matching. Histogram matching is then applied to normalize illumination differences between the cameras, creating a seamless stitched frame.
For detection, the system employs the Viola-Jones cascade detector for faces. Eye detection uses a specialized template matching approach focused on specular highlights (NIR glints), which provides robustness in lower-resolution video frames.
Matching and Fusion
The system extracts features and calculates match scores for both modalities:
- Face Matching: Utilizes the "Eigenface" approach with a Mahalanobis cosine metric.
- Iris Matching: Employs a modified Daugman algorithm to calculate normalized Hamming distances.
The core contribution lies in the fusion of these scores. The researchers compared several strategies, including multi-sample (taking the best score across frames), rank-level (Borda Count), and score-level fusion. Weighted score-level fusion (SumBoth), which performs a weighted summation of normalized face and iris scores, proved to be the most effective method.
Experimental Setup
The study utilized a genuine multi-modal dataset comprising 1,886 IOM video sets from 363 unique subjects. Unlike chimeric datasets, this data captures real-world dependencies between face and iris traits. The gallery consisted of high-quality, manually screened images, while the probes were the "on-the-move" video frames captured as subjects walked through the portal.
Results
The fusion approach significantly outperformed single-modality recognition. While face recognition alone (MinFace) achieved a 62.6% rank-one recognition rate and iris recognition (MinIris) reached 87.8%, the SumBoth fusion method achieved 93.2%. This 5.4% absolute improvement over the best single modality highlights the resilience of multi-modal systems; if one trait is partially obscured, the other can still facilitate a confident match.
Discussion and Future Work
Despite the success, limitations remain. Motion blur and defocus in video frames occasionally hindered matching. Furthermore, the Viola-Jones detector struggled with partial faces when subjects were not fully within the camera's field of view. Future research will focus on incorporating "quality metrics" to automatically weight frames based on sharpness and exploring more sophisticated partial-face matching algorithms to further enhance robustness.
Summary updated