TL;DR
This paper introduces a novel approach for EEG-based emotion recognition by transforming 1D EEG signals into 2D "EEG feature maps" that preserve spatial topology. They then employ a Feature Pyramid Network (FPN) combined with Long Short-Term Memory (LSTM) to effectively extract both spatial and temporal features, achieving high accuracy in recognizing valence and arousal states.
What Problem Does This Paper Address?
Recognizing human emotions from Electroencephalography (EEG) signals is a challenging but important area for human-computer interaction. Traditional EEG analysis often treats signals as 1D time series, neglecting the crucial spatial information encoded by the arrangement of electrodes on the scalp. This spatial topology is believed to be closely related to different brain functions and, consequently, to emotional states. Existing deep learning methods, while improving accuracy, often struggle to effectively integrate this spatial information with temporal dynamics, leading to suboptimal performance. The paper aims to bridge this gap by developing a method that explicitly captures and leverages both spatial and temporal features from EEG for more robust emotion recognition.
Methods
The core of this work lies in two main innovations:
- EEG Feature Map Generation: To preserve spatial topology, the authors first convert the 1D EEG signals into 2D "EEG feature maps." This involves:
- Azimuth Equidistant Projection (AEP): This projection method transforms the 3D spatial arrangement of EEG electrodes into a 2D plane, retaining topological relationships.
- Feature Extraction: For three key frequency bands (alpha, beta, and gamma), the average power, variance power, and standard deviation power are extracted as feature data.
- Image Generation: These features are then used as the R, G, and B channels to create a color EEG feature map. BiCubic interpolation is used to fill in missing pixel data between electrodes.
- FPN-LSTM Network: A Feature Pyramid Network (FPN) is integrated with an LSTM network for feature extraction.
- FPN: This network is designed to enhance feature extraction by combining features from different scales. It has a bottom-up path for generating features and a top-down path for enhancing them, with lateral connections to fuse information. This helps in better representing features across various dimensions of the input image.
- LSTM: The FPN's output, which captures multi-scale spatial features, is then fed into an LSTM network to extract temporal dependencies from the EEG feature maps.
- Channel Weighting: A novel idea of assigning differential weights to EEG channels is introduced. Channels with stronger known emotional correlation (e.g., AF3, F3, F7, FC5, T7) are given higher weights, while others receive lower weights, aiming to focus on more informative signals.
Datasets
The study utilizes the DEAP (Database for Emotion Analysis using Physiological Signals) dataset. This dataset contains EEG signals from 32 healthy subjects (16 male, 16 female) who were exposed to various stimuli. The data includes 40 trials per subject, with each trial containing 40 channels of EEG data sampled at 512 Hz.
Key Insights and Ideas
- Spatial Topology Matters: Transforming 1D EEG into 2D feature maps that preserve spatial relationships significantly improves emotion recognition performance compared to methods that ignore this aspect.
- FPN for Multi-Scale Feature Fusion: The FPN architecture is effective in capturing and integrating features from different spatial scales within the EEG feature maps, leading to richer representations.
- Synergy of Spatial and Temporal Models: Combining FPN (for spatial features) with LSTM (for temporal features) creates a powerful model that leverages both dimensions of EEG data for emotion recognition.
- Channel Weighting Enhances Focus: Strategically assigning weights to EEG channels based on their known emotional correlation can boost recognition accuracy by emphasizing more relevant signals and downplaying less informative ones.
- Individual Differences in Emotion: The generated EEG feature maps visually demonstrate that even with the same stimulus, men and women can exhibit distinct emotional responses, highlighting the importance of personalized or robustly generalized models.
Key Results
The proposed FPN-LSTM method achieved impressive results on the DEAP dataset:
- Valence Recognition: 90.05% accuracy.
- Arousal Recognition: 90.84% accuracy.
These results are competitive and, in some aspects, superior to previous studies using the DEAP dataset, particularly when compared to methods relying solely on 1D EEG data. The paper also shows that using average power as the feature for EEG feature maps yielded better results than variance or standard deviation power. The channel weighting strategy with a weight of 0.8 for strongly correlated channels proved optimal.
Limitations and Future Work
- Influence of Other Physiological Signals: The authors acknowledge that the presence or absence of other physiological signals (like EOG, EMG) could influence the recognition accuracy of valence and arousal, suggesting that multi-modal approaches might be beneficial.
- Labeling Methods: Differences in how emotions are labeled (e.g., discrete vs. dimensional) could impact results.
- Channel Selection: While the paper explores channel weighting, the choice of which channels to weight and the specific weights could be further optimized. The authors suggest that weakening data from 27 channels might have slightly impacted valence recognition compared to a reference method.
- Future Work: The paper implicitly suggests exploring multi-modal fusion and more sophisticated channel selection or weighting strategies as avenues for future research.
Summary updated