TL;DR
This paper introduces a novel multimodal biometric system that fuses face and finger vein features using a convolutional neural network (CNN) with a self-attention mechanism. By combining these two biometrics at the feature layer, the system achieves high accuracy (over 98.4%) and robustness, addressing the limitations of single-modal biometric systems.
What Problem Does This Paper Address?
In our increasingly digital world, secure and accurate identity verification is paramount. Traditional methods like passwords are prone to theft and impersonation. Biometrics offer a more secure alternative, but single biometric systems (like face or fingerprint recognition) struggle with diverse real-world scenarios. For instance, face recognition is hindered by masks, and fingerprint recognition can be affected by environmental factors. This paper tackles the challenge of improving biometric system accuracy and security by proposing a multimodal approach that fuses face and finger vein biometrics, leveraging the strengths of both.
Methods
The core of this work is a two-channel CNN framework designed for feature layer fusion. Here's a breakdown of the key components:
- Feature Extraction: Both face and finger vein images are processed through separate CNN branches. The paper utilizes pre-trained AlexNet and VGG-19 architectures (specifically, their convolutional and pooling layers) as feature extractors.
- Self-Attention Mechanism: A novel self-attention mechanism is employed to dynamically weigh the importance of features from each modality (face and finger vein). This helps the system focus on the most discriminative information from each source.
- Feature Fusion: The weighted features from both modalities are concatenated. This combined feature vector is then processed through a "Fusion Block" (another set of convolutional layers) to extract deeper, fused features. The paper also mentions cascading these fused features with the original extracted features to prevent information loss.
- Classification: Finally, a fully connected layer is used for the classification and identification of individuals.
The use of a self-attention mechanism for weighting features and the feature-layer fusion approach are the novel aspects of their methodology.
Datasets
The study utilizes publicly available datasets for training and evaluation:
- CASIA-WebFace: A widely used dataset for face recognition.
- FV-USM: A Malaysian Polytechnic finger vein dataset.
- SDUMLA-FV: A finger vein dataset from Shandong University.
The paper mentions data augmentation techniques were applied to the finger vein datasets to mitigate the issue of limited data.
Key Insights and Ideas
- Complementary Strengths: The fusion of face (an external biometric) and finger vein (an internal biometric) is a smart choice. Faces are easily recognizable but can be obscured, while finger veins are difficult to forge but require closer proximity. Combining them offers a more robust solution.
- Feature-Layer Fusion is Key: The paper argues that fusing features at the "feature layer" (rather than earlier or later stages) captures richer, more discriminative information, leading to higher accuracy.
- Self-Attention for Smarter Fusion: The self-attention mechanism is crucial for adaptively determining which biometric modality is more reliable in a given situation, leading to a more intelligent fusion process.
- CNNs are Powerful Feature Extractors: The successful application of AlexNet and VGG-19 demonstrates the effectiveness of deep convolutional neural networks for extracting complex features from both face and finger vein images.
Key Results
The proposed bimodal fusion method achieved impressive results:
- High Accuracy: The recognition accuracy for both AlexNet and VGG-19 based feature extraction exceeded 98.4% in multimodal experiments. The lowest reported accuracy was 98.84%, and the highest was 99.98%.
- Improved Performance: Compared to single-modal identification (which showed accuracies as low as 45.61%), the multimodal approach significantly boosts performance.
- Excellent ROC and P-R Curves: The ROC curves show AUC values of 1 for both micro and macro averages, indicating near-perfect classification. The P-R curves also demonstrate strong performance with average AP values above 0.88.
Limitations and Future Work
- Computational Requirements: The current research is limited to systems with high-performance GPUs and cannot be directly deployed on mobile embedded terminals.
- Future Work: The authors plan to "lightweight" the network results of bimodal fusion and port the model to embedded terminals to expand its practical applicability.
Summary updated