TL;DR
This paper introduces SignEEG v1.0, a novel multimodal dataset combining electroencephalography (EEG) and handwritten signatures for enhanced biometric systems. By integrating a forgery-resistant physiological modality (EEG) with a behavioral one (signatures), the dataset aims to significantly improve the robustness and reliability of user identification and verification against sophisticated forgeries.
What Problem Does This Paper Address?
Handwritten signatures are a widely adopted biometric modality due to their convenience and legal recognition. However, they are vulnerable to sophisticated forgery attempts, posing a significant security challenge. To address this, researchers are exploring multimodal approaches that combine signatures with other biometric traits. Electroencephalography (EEG), which captures unique brain activity patterns, is a promising candidate due to its inherent resistance to replication. The challenge lies in effectively integrating these modalities and having accessible, comprehensive datasets to develop and evaluate such systems. Existing research in this area is scarce, often limited to single paradigms, and lacks publicly available data, hindering further investigation.
Methods
The study introduces the SignEEG v1.0 dataset, collected from 70 participants. The data acquisition involved three distinct paradigms:
- Mental Imagery (Signature Image): Participants closed their eyes and visualized their signature.
- Mental Imagery (Signature Trajectory): Participants closed their eyes and mentally simulated the motor actions of drawing their signature.
- Physical Execution: Participants opened their eyes and physically drew their signature on a Wacom tablet.
EEG signals were captured using an Emotiv Insight headset, while signature data was recorded using a Wacom One display pen tablet. The data underwent pre-processing using EEGLAB, including epoching, bandpass filtering, baseline removal, and Independent Component Analysis (ICA) for artifact removal. For evaluation, various machine learning models (Gaussian NB, KNN, Linear SVM, Random Forest, Gradient Boost, XGBoost) were employed, utilizing statistical and wavelet features from EEG, as well as statistical and dynamic features from Wacom signatures.
Datasets
The primary contribution of this paper is the introduction of the SignEEG v1.0 dataset. This dataset comprises:
- EEG signals and corresponding handwritten signatures.
- Data from 70 healthy participants.
- Collected over several months.
- Includes three distinct experimental paradigms (mental imagery of signature image, mental imagery of signature trajectory, and physical signature execution).
- Data is available in EDF (for EEG) and CSV (for signatures) formats, along with pre-processed MATLAB structures.
- The dataset is publicly available on Zenodo under a Creative Commons Attribution 4.0 International License.
Key Insights and Ideas
- Multimodality as a Forgery Countermeasure: The core insight is that combining a behavioral modality (signatures) with a physiological, forgery-resistant modality (EEG) significantly enhances biometric system robustness against sophisticated forgeries.
- EEG's Resistance to Replication: EEG signals, representing unique brain activity, are inherently difficult to replicate, making them a strong complement to signatures.
- Paradigm Diversity is Crucial: The dataset's inclusion of three distinct paradigms (mental imagery and physical execution) allows for a more comprehensive exploration of brain-behavioral correlations in signature-based biometrics.
- Baseline Performance: The study establishes a baseline performance for various machine learning models, demonstrating the effectiveness of multimodal approaches even with limited sample sizes.
Key Results
The study reports promising results for multimodal biometric systems. For identification, models combining EEG and Wacom signature features achieved high accuracy, with XGBoost reaching 98.94% when using statistical and wavelet EEG features combined with Wacom features. For verification, Random Forest and Gradient Boost achieved the highest accuracy at 94.89% and 93.40% respectively, using a combination of statistical, wavelet, and Wacom features. The results consistently show that multimodal approaches significantly outperform unimodal ones, particularly for signature-based biometrics.
Limitations and Future Work
The authors acknowledge that the longitudinal temporal dynamics of EEG and handwritten signatures are yet to be explored. Future work will focus on intensively studying these temporal dynamics through prolonged, longitudinal data collection from the same subjects. This approach aims to reveal how these biometric modalities evolve and interact over time, potentially leading to further advancements in multimodal biometric system accuracy and reliability. Additionally, the paper notes that data from 5 subjects were set aside for anticipated future research, suggesting potential for further validation or exploration.
Summary updated