TL;DR
This paper introduces SSAS, a novel framework for EEG-based emotion recognition that tackles the challenge of inter-subject variability. By employing a source selection network (SS) and an adversarial strategy network (AS), SSAS intelligently identifies and down-weights unreliable data sources, leading to significantly improved emotion recognition accuracy, especially in heterogeneous datasets.
What Problem Does This Paper Address?
Electroencephalography (EEG) is a promising tool for understanding emotions, but building accurate emotion recognition models that work across different individuals (cross-subject) is a major hurdle. This is primarily due to "inter-individual variability" – everyone's brain activity is unique. Existing methods often struggle with "negative transfer," where knowledge learned from one subject's data can actually harm the model's performance on another subject. This can happen when the source data is noisy, irrelevant, or simply too different from the target subject's data. The paper aims to address this by not just adapting models to new subjects, but by first carefully selecting which existing subject data is most beneficial for learning.
Methods
SSAS is built upon two core modules:
- Source Selection Network (SS): This module is the key innovation. Instead of relying solely on traditional similarity metrics (like MMD) between raw data, the SS network uses the domain labels (which subject the data came from) to reverse-engineer the domain adaptation process. It aims to make it harder for the model to classify data based on its source subject, forcing it to learn domain-invariant representations. This process helps identify which source domains are truly transferable and which might cause negative transfer.
- Adversarial Domain Adaptation Network (AS): This module takes the insights from the SS network and performs the actual emotion recognition. It uses adversarial learning to align the feature distributions between the selected source domains and the target domain, making the model more robust. It also incorporates Gaussian noise to improve the model's resilience to variations.
The overall strategy is to first intelligently filter out problematic source data before applying advanced domain adaptation techniques.
Datasets
The proposed SSAS method was evaluated on three publicly available EEG emotion datasets:
- SEED: Records EEG from 15 subjects watching emotional movie clips.
- SEED-IV: Similar to SEED but uses four emotion categories ('happiness,' 'sadness,' 'fear,' and 'neutral') and a different experimental setup.
- HBUED: A larger dataset with 50 subjects, using video stimuli for emotion induction.
The paper notes that SEED and SEED-IV provide precomputed differential entropy (DE) features, which were used as input.
Key Insights and Ideas
- Domain Labels as a "Reverse Engineering" Tool: The most significant insight is using domain labels (subject IDs) to guide the source selection process. This is a departure from traditional methods that rely solely on data similarity metrics.
- Negative Transfer is a Major Bottleneck: The paper strongly emphasizes that negative transfer is a critical issue in cross-subject EEG analysis, and SSAS directly targets this by identifying and down-weighting detrimental source data.
- Heterogeneity Benefits SSAS: The source selection mechanism of SSAS proves more impactful on datasets with higher inter-subject variability (like SEED-IV), suggesting its strength lies in handling diverse and potentially noisy source data.
- Iterative Source Weighting: The SS module iteratively adjusts weights for source domains, progressively emphasizing those that contribute positively to target domain learning and down-weighting those that don't.
Key Results
SSAS demonstrated significant improvements across all tested datasets:
- SEED: Achieved the highest accuracy (91.97%) in the best session, outperforming 20 existing methods and showing an average improvement of at least 2.05% over state-of-the-art methods.
- SEED-IV: Reached the highest accuracy (77.99%) in one session, with an approximate 33.4% increase over a baseline and improvements of 1.18% and 5.07% over other methods in different sessions.
- HBUED: Showed substantial improvements in valence, arousal, and four-class classification tasks compared to baseline methods.
- Ablation Studies: Removing individual components of SSAS (like MDC loss, MMD loss, or the adversarial component) led to noticeable drops in accuracy, highlighting the contribution of each part. Notably, removing the SS module itself resulted in a 4.61% and 7.02% accuracy drop on SEED and SEED-IV, respectively.
Limitations and Future Work
- Computational Overhead: The source selection stage adds computational complexity. While beneficial for heterogeneous datasets, its marginal benefit on datasets with uniform source quality is acknowledged.
- Future Work: The authors suggest exploring lighter selection mechanisms or adaptive gating to reduce overhead in near-homogeneous settings. They also propose learning more fine-grained source weights.
Summary updated