TL;DR
This paper introduces SignEEG v1.0, a novel multimodal dataset combining electroencephalography (EEG) and handwritten signatures to enhance biometric system robustness against sophisticated forgeries. By integrating physiological (EEG) and behavioral (signature) modalities, the authors demonstrate significantly improved identification and verification accuracy, paving the way for more secure and reliable biometric solutions.
What Problem Does This Paper Address?
Handwritten signatures are a widely used biometric modality due to their convenience and legal recognition. However, they are highly susceptible to forgery attempts, which can be difficult to detect. To combat this, researchers are exploring multimodal approaches that combine signatures with other, more forgery-resistant modalities. Electroencephalography (EEG), which captures unique brain activity patterns, is a promising candidate due to its inherent difficulty to replicate. The challenge lies in effectively integrating these two modalities and creating a comprehensive dataset that supports research in this area. Existing work in this domain is limited by scarce data and a lack of standardized experimental paradigms.
Methods
The study proposes a multimodal biometric framework that combines EEG and handwritten signatures. The core of their contribution is the creation of the SignEEG v1.0 dataset, which includes data from 70 participants. The data collection involved three distinct experimental paradigms:
- Mental Imagery (Signature Image): Participants close their eyes and visualize their signature.
- Mental Imagery (Signature Trajectory): Participants close their eyes and mentally simulate the motor movements required to draw their signature.
- Physical Execution: Participants physically draw their signature on a Wacom tablet with their eyes open.
EEG data was acquired using an Emotiv Insight headset, and signature data was captured using a Wacom One pen tablet. The data underwent a standard EEG pre-processing pipeline, including epoching, bandpass filtering (0.1-60Hz), baseline removal, and artifact removal using Independent Component Analysis (ICA). For signature data, features were extracted by calculating statistical and dynamic measures from raw pen data. Various machine learning models (Gaussian NB, KNN, Linear SVM, Random Forest, Gradient Boost, XGBoost) were then trained and evaluated on unimodal (EEG or Wacom) and multimodal (EEG + Wacom) feature sets for both identification and verification tasks.
Datasets
The paper introduces the SignEEG v1.0 dataset, a new benchmark for multimodal biometric research. This dataset comprises:
- 70 healthy participants.
- EEG data collected using an Emotiv Insight headset.
- Handwritten signature data captured on a Wacom One pen tablet.
- Data collected across three experimental paradigms: mental imagery of signature image, mental imagery of signature trajectory, and physical signature execution.
The dataset is available in EDF (EEG), CSV (signature), and MATLAB (.mat) formats.
Key Insights and Ideas
- Multimodality is Key: Combining physiological (EEG) and behavioral (signature) modalities significantly boosts biometric system robustness and accuracy, particularly against forgery.
- EEG's Forgery Resistance: EEG signals are inherently difficult to forge, making them a strong complement to signature-based biometrics.
- Diverse Paradigms Matter: The three distinct experimental paradigms (visualizing, mentally simulating, and physically drawing) capture different aspects of cognitive and motor processes related to signature generation, providing a richer dataset.
- XGBoost and Random Forest Shine: These models consistently perform well across various feature sets and tasks, especially when combining EEG and signature data.
- Data Availability is Crucial: The release of the SignEEG v1.0 dataset is a significant contribution, enabling further research and development in this domain.
Key Results
The study demonstrates that multimodal approaches significantly outperform unimodal ones. For identification, combining statistical EEG features with Wacom signature features yielded the highest accuracy, with XGBoost achieving 98.68% and Random Forest reaching 97.63%. For verification, similar trends were observed, with Random Forest achieving 98.94% when using statistical, wavelet, and Wacom features combined. Even with limited sample sizes, the multimodal approach achieved high reliability.
Limitations and Future Work
The authors acknowledge that the longitudinal temporal dynamics of EEG have not been explored. Future work will focus on extending data collection to capture these temporal dynamics over prolonged periods to understand how biometric modalities evolve and interact over time, aiming to further improve system accuracy and reliability. Additionally, while the dataset is comprehensive, the authors note that 5 subjects' data was set aside for future robustness analysis, suggesting potential for further exploration with this subset.
Summary updated