TL;DR
This paper introduces Attribute-based Deep Periocular Recognition (ADPR), a novel deep learning framework that enhances periocular recognition by jointly learning and fusing "soft biometrics" (like gender and ethnicity) with traditional periocular features. This approach significantly improves recognition accuracy, especially in challenging "wild" environments where faces are partially occluded.
What Problem Does This Paper Address?
Traditional facial recognition systems often struggle when faces are partially obscured, a problem exacerbated by recent events like the COVID-19 pandemic, which necessitated widespread mask-wearing. Periocular recognition, focusing on the region around the eyes, emerged as a promising alternative. However, periocular recognition in unconstrained, "wild" environments remains challenging due to variations in distance, illumination, and occlusions like hair and glasses. Existing methods often rely solely on the visual features of the periocular region. This paper addresses this by hypothesizing that semantic information, akin to how humans recognize faces by considering attributes like gender and ethnicity, can also benefit automated periocular recognition. They aim to integrate these "soft biometrics" into a deep learning framework to boost accuracy in these difficult scenarios.
Methods
The core of the proposed method is the Attribute-based Deep Periocular Recognition (ADPR) framework, an end-to-end deep learning architecture. It utilizes a shared convolutional neural network (CNN) backbone (based on VGG-16 pre-trained on ImageNet) to extract features. These features are then fed into two separate branches:
- Periocular Recognition (PR) Branch: This branch is dedicated to classifying periocular images for identity recognition.
- Soft Biometric Classifier Branch: This branch is trained to predict soft biometric attributes (e.g., gender, ethnicity, skin color) from the periocular region.
Crucially, the features from these two branches are fused in a Joint Periocular Recognition (JPR) block. This fusion happens not just at the prediction stage but also during the training process. The framework is trained using a combined loss function that backpropagates errors from both the periocular recognition task and the soft biometric classification task. This joint learning and feature fusion allow the soft biometric information to directly enhance the discriminative power of the periocular recognition features.
For open-world scenarios (where test subjects are not seen during training), the authors also propose an ADPR-based Siamese network architecture using a contrastive loss function. This allows the model to learn a similarity metric that generalizes to unseen identities.
Datasets
The authors utilized several publicly available datasets for their experiments:
- UBIPr: Used for training the ADPR model in the open-world setting.
- UBIRISV2: Used for evaluating both open-world and closed-world settings.
- FRGC (Face Recognition Grand Challenge): Used for evaluating both open-world and closed-world settings. This dataset required cropping periocular regions from full facial images.
The datasets were annotated with soft biometrics such as gender and ethnicity.
Key Insights and Ideas
- Soft Biometrics as a Powerful Auxiliary: The paper strongly demonstrates that semantic "soft biometrics" (gender, ethnicity) are not just ancillary but can significantly boost the performance of periocular recognition, especially in challenging conditions.
- Joint Learning is Key: Simply predicting soft biometrics alongside periocular features isn't enough. The true power comes from fusing these features and jointly optimizing a combined loss function during training, allowing the network to learn how soft biometrics inform periocular identity.
- End-to-End Fusion Enhances Discriminability: The end-to-end training of the fused network allows the soft biometric information to directly influence and improve the discriminative features extracted for periocular recognition.
- Generalization to Unseen Subjects: The Siamese network adaptation shows promise for open-world verification, enabling generalization to subjects not encountered during training, a critical aspect for real-world biometric systems.
Key Results
- Open-World Performance: The proposed ADPR (JPR) consistently outperformed the ADPR (PR) (without soft biometrics) and other state-of-the-art methods (SCNN, PPDM, Texton) in open-world verification, showing improvements in Equal Error Rate (EER). For instance, on the UBIRISV2 dataset, the EER was improved by 3% over the second-best method.
- Closed-World Performance: In closed-world settings, ADPR achieved very low EERs (e.g., 1.73% on UBIRISV2, 0.58% on FRGC), significantly outperforming other methods. Rank-1 accuracy reached over 92.68% on UBIRISV2.
- Soft Biometric Prediction: The fusion of periocular features also improved soft biometric prediction accuracy by at least 1.5% compared to using only soft biometrics.
- Effect of Multiple Soft Biometrics: Using multiple soft biometrics (gender and ethnicity) further improved performance compared to using a single attribute, indicating that richer semantic information leads to better recognition.
Limitations and Future Work
- Dataset Dependency: While the paper evaluates on multiple datasets, the performance might still be influenced by the specific characteristics of these datasets.
- Computational Cost: Deep learning models, especially those with multiple branches and fusion layers, can be computationally intensive. The paper mentions using pre-trained backbones to mitigate this.
- Future Work: The authors suggest exploring more soft biometric attributes and investigating their framework for other biometric modalities or multimodal fusion scenarios. The Siamese network approach for open-world settings is also a promising direction for future research.
Summary updated