Abstract
A discriminative and robust feature—the kernel-enhanced informative Gabor feature—is proposed for face recognition. Mutual information is applied to select a set of informative and nonredundant Gabor features, which are then further enhanced by kernel methods for recognition. Compared with top-performing methods from the 2004 Face Verification Competition (FVC2004), our approach demonstrates a clear advantage in accuracy, computational efficiency, and memory cost. The proposed method has been fully tested on the FERET database using the standard evaluation protocol, showing significant improvements across multiple test sets. While classical Gabor wavelet-based approaches utilize a massive number of features, our method requires less than 4 milliseconds to retrieve a few hundred features. Due to the substantially reduced feature dimension, only 4 seconds are required to recognize 200 face images. This work also unifies different Gabor filter definitions and proposes a training sample generation algorithm to mitigate the effects of unbalanced sample sizes across classes.
Introduction
Gabor filters are a staple in computer vision because they mimic the way human visual neurons process information, making them highly robust against changes in lighting, facial expression, and head orientation. However, they suffer from a massive "dimensionality curse."
Standard methods convolve a face image with a bank of 40 Gabor filters (5 scales and 8 orientations). For a modest $64 \times 64$ pixel image, this generates 163,840 features. Processing, storing, and comparing these massive feature vectors is computationally expensive and memory-intensive. Previous attempts to solve this, such as using AdaBoost, often select features that are individually strong but highly redundant, leading to sub-optimal efficiency.
Methodology
The proposed pipeline consists of two primary stages: InfoGabor for feature selection and Generalized Discriminant Analysis (GDA) for enhancement.
1. Difference Space Modeling
Instead of analyzing raw images directly, the problem is transformed into "difference space." The system evaluates the difference between two images and classifies that difference as either intrapersonal (the same person) or extrapersonal (different people).
2. Conditional Mutual Information (CMI)
To select the most effective features, the authors utilize CMI from Information Theory. Unlike standard Mutual Information, CMI measures how much new information a feature provides given the features already selected. This explicitly penalizes redundancy, ensuring each selected feature adds unique value.
3. Binarization and Kernel Enhancement
To make the complex mathematics of CMI tractable, Gabor features are binarized based on a threshold calculated from the mean of the training classes. Once the top ~200 features are selected, Generalized Discriminant Analysis (GDA) is applied. This non-linear kernel method projects the selected features into a lower-dimensional space where different subject classes are maximally separated, facilitating classification via a simple Nearest Neighbor (NN) classifier.
Experimental Results
Datasets and Evaluation
The method was benchmarked using the FERET Database. The authors utilized a subset of 600 images for initial testing and the full FERET evaluation protocol (Gallery of 1,196 images) for final validation. Evaluation was performed across four specific probe sets: Fb (expression change), Fc (lighting change), Dup I (images taken within 1 week), and Dup II (images taken at least 1 year apart).
Performance and Efficiency
- Massive Compression: Feature dimensionality was reduced from 163,840 to 200, representing an 819x reduction.
- Computational Speed: Retrieving selected features takes less than 4 milliseconds, and recognizing 200 face images takes only 4 seconds. Training the GDA subspace dropped from 275 seconds to just 6 seconds.
- Accuracy: On the difficult Dup II set, the method achieved 65.38% accuracy, significantly outperforming Elastic Graph Matching (52.1%) and standard LDA (20.9%).
Discussion
Redundancy and Feature Localization
The core insight of this work is that 99% of Gabor features are unnecessary. By focusing on conditional information, the authors prove that 200 carefully chosen features can outperform 160,000 raw ones. Visualization of these 200 features shows they cluster heavily around stable facial landmarks: the eyes, eyebrows, nose, and chin.
Frequency and Orientation Bias
The selection algorithm naturally preferred low-frequency filters and horizontal orientations. This suggests that the broad structure of the face and horizontal features (like the line of the eyes or mouth) are more discriminative than fine-grained vertical details. Furthermore, the authors demonstrated that Gabor features in difference space do not follow a Gaussian distribution, justifying the use of distribution-agnostic Information Theory over traditional statistical methods.
Conclusion and Future Work
While highly effective, the current method relies on binarization, which may lose nuanced information. Future work aims to extend the CMI selection process to continuous variables, potentially using Gaussian Mixture Models (GMMs) to represent feature distributions more accurately without the need for thresholding. This could further refine the selection process and improve recognition accuracy in even more challenging environments.
Summary updated