TL;DR
This paper proposes a novel multimodal biometric system that fuses electroencephalography (EEG) signals with dynamic signature data for enhanced person authentication on mobile devices. By combining brain activity patterns with the nuances of signature strokes, the system achieves a remarkable 98.78% accuracy in identification and a low 3.75% False Acceptance Rate (FAR) in verification, significantly outperforming unimodal approaches.
What Problem Does This Paper Address?
Traditional biometric systems, whether based on physiological traits (like fingerprints or iris scans) or behavioral patterns (like passwords or signatures), are susceptible to various attacks, including spoofing, obfuscation, and covert manipulation. While passwords and signatures are familiar, they can be easily forged or forgotten. Existing biometric systems often struggle with these vulnerabilities, especially in the context of increasingly interconnected and mobile technologies where robust authentication is paramount. The paper aims to address this by developing a more secure and difficult-to-forge authentication system by integrating two distinct biometric modalities: dynamic signatures and EEG signals.
Methods
The core of this research lies in a novel multimodal framework that simultaneously captures dynamic signature data and EEG signals from users signing on a mobile device.
- Data Acquisition: EEG signals were recorded using an Emotiv EPOC+ headset, while dynamic signature data (x,y coordinates and pressure) was captured via a custom GUI on a Samsung Galaxy Note 9.
- Feature Extraction:
- EEG (FD): Discrete Fourier Transform (DFT) was employed to extract features from EEG signals, focusing on Gamma band waves across all 14 channels.
- Dynamic Signatures (Fs): Features included the raw time-series of signature trajectory (T) and writing direction (W), calculated as the slope between neighboring points on the signature path.
- Classification: A Bi-directional Long Short-Term Memory Neural Network (BLSTM-NN) was used as the classifier for both unimodal (signature-only or EEG-only) and multimodal approaches. BLSTM-NNs are chosen for their ability to model temporal data in both forward and backward directions.
- Multimodal Fusion: Decision fusion techniques, specifically Sum Rule and Borda Count, were applied to combine the outputs of the unimodal BLSTM-NN classifiers. The Borda Count method proved most effective.
- Verification: A threshold-based approach was used for verification, where the system compares a user's claimed identity against reference templates and makes a decision based on a predefined acceptance threshold.
Datasets
- Custom Dataset: The study collected data from 58 healthy volunteers (33 genuine users and 25 forgers).
- Genuine users provided 10 signatures each, resulting in 330 signature and EEG samples.
- Forgers attempted to imitate genuine signatures twice each, generating 1650 forged signature samples.
- Publicly Available Datasets (for additional analysis):
- Physionet: A 64-channel EEG dataset of 109 users.
- MCYT DB1: A dynamic signature database of 100 individuals.
Key Insights and Ideas
- Synergy of Brain and Hand: The fundamental insight is that the act of signing stimulates specific patterns in brain activity (EEG), and these patterns are distinct enough to be used as a biometric. Combining this with the physical act of signing creates a much more robust and difficult-to-forge system.
- EEG as a "Digital Fingerprint": The paper demonstrates that while a skilled forger might imitate the visual appearance of a signature, they cannot easily replicate the underlying neural activity associated with the genuine signing process. This makes EEG a powerful complement to dynamic signatures.
- BLSTM-NN's Temporal Modeling Power: The choice of BLSTM-NN is crucial, as it effectively captures the temporal dependencies inherent in both signature trajectories and EEG signal sequences, which are vital for accurate classification.
- Borda Count for Fusion: The Borda Count fusion method emerged as superior, suggesting that its ranking-based approach is more effective at integrating diverse biometric signals compared to simple summation or maximum selection.
Key Results
- Person Identification: The multimodal system achieved a peak accuracy of 98.78% using the Borda Count fusion technique.
- Unimodal Dynamic Signature: 96.36% accuracy.
- Unimodal EEG Signals: 97.57% accuracy (using Gamma band waves from all channels).
- Person Verification:
- False Acceptance Rate (FAR): 3.75% with 100% True Positive Rate (TPR) for forgery attempts, using the Borda Count fusion.
- Half Total Error Rate (HTER): 1.87% for the multimodal system.
- Unimodal EEG: 11.25% HTER.
- Unimodal Dynamic Signature: 7.45% HTER.
- Comparison with Other Datasets: The proposed BLSTM classification scheme showed competitive or improved performance on Physionet and MCYT DB1 datasets compared to existing methods.
Limitations and Future Work
- Unimodal Errors: The paper acknowledges that unimodal systems can make errors. For signatures, similar writing styles between users can cause confusion. For EEG, lack of concentration or motion artifacts (due to body movements) can lead to misclassification. However, the multimodal approach often compensates for these individual modality weaknesses.
- Future Work:
- Robustness to Stress/Fatigue: Investigating the stability of these signals during stressful situations or over extended periods.
- Additional Sensors: Exploring the incorporation of Inertial Measurement Units (IMUs) for movement detection to further enhance the system.
- Advanced Fusion and NN Architectures: Employing more sophisticated multi-classifier combination methods and neural network topologies.
- Real-world Deployment: The authors suggest that such systems could become feasible for a wide range of security applications, including smart homes and healthcare, with future research.
Summary updated