TL;DR
This paper introduces SDA-DDA, a novel semi-supervised domain adaptation framework for EEG-based emotion recognition. It tackles the significant challenge of individual variability in EEG signals by dynamically aligning both marginal and conditional probability distributions between source and target domains, coupled with a confidence-based pseudo-label filtering mechanism for improved robustness.
What Problem Does This Paper Address?
Affective Brain-Computer Interfaces (aBCIs) aim to recognize human emotional states using physiological signals like EEG. However, a major hurdle is the inherent variability in EEG data across individuals, making it difficult to build models that generalize well. This "individual variability" means that a model trained on one person's EEG data might perform poorly on another's. Traditional domain adaptation methods often struggle with this, especially when the target domain (new individuals) has limited or no labeled data. The paper specifically highlights that while many methods focus on aligning the overall data distribution (marginal probability distribution), they often overlook the nuances within specific emotion categories (conditional probability distribution), which is crucial for accurate, personalized emotion recognition.
Methods
The core of SDA-DDA lies in its novel Semi-supervised Domain Adaptation with Dynamic Distribution Alignment (SDA-DDA) framework. It comprises four key modules:
- Marginal Probability Distribution (MPD) Alignment: Uses Maximum Mean Discrepancy (MMD) to measure and minimize the divergence between the overall distributions of source and target EEG data.
- Conditional Probability Distribution (CPD) Alignment: Introduces Conditional Maximum Mean Discrepancy (CMMD) to specifically align the distributions of features within each emotion category. This is crucial for capturing subtle differences between emotions across individuals.
- Dynamic Distribution Alignment: Unlike static alignment methods, SDA-DDA dynamically adjusts the influence of MPD and CPD alignment throughout training. This allows the model to adapt to changing data distributions and better capture the complex relationships between them.
- Pseudo-label Confidence Filtering: In the semi-supervised setting, pseudo-labels are generated for unlabeled target data. A dynamic confidence threshold strategy is employed to filter these pseudo-labels, retaining only the most reliable ones. This mechanism adapts over training epochs, starting with a low threshold to include more data and gradually increasing it to focus on high-confidence samples, thereby improving the quality of learned features and reducing noise.
Datasets
The proposed SDA-DDA framework was evaluated on three publicly available EEG emotion recognition datasets:
- SEED: Contains EEG data from 15 participants across three sessions, with emotional stimuli from Chinese film clips.
- SEED-IV: Features data from 15 subjects over three sessions, using film clips to elicit happiness, sadness, neutrality, or fear.
- DEAP: Includes EEG data from 32 participants who watched music videos and rated their emotional responses.
Key Insights and Ideas
- CPD Alignment is Crucial: The paper strongly emphasizes that aligning conditional probability distributions (within emotion categories) is as important, if not more so, than aligning marginal distributions for robust cross-subject emotion recognition.
- Dynamic Adaptation is Key: Static alignment methods can be insufficient. Dynamically adjusting the weights of MPD and CPD alignment based on training progress allows the model to better adapt to the target domain.
- Pseudo-label Quality Matters: In semi-supervised learning, the quality of generated pseudo-labels significantly impacts performance. A dynamic confidence filtering mechanism is essential for filtering out noisy or incorrect pseudo-labels.
- SDA-DDA is Computationally Efficient: The framework achieves high accuracy with a relatively low computational load (3.01 million FLOPs) and fast inference speed (20 ms per batch), making it suitable for real-time applications.
Key Results
The SDA-DDA model demonstrated superior performance across the evaluated datasets:
- SEED Dataset: Achieved an accuracy of 87.27% ± 07.55% in cross-subject single-session leave-one-subject-out validation.
- SEED-IV Dataset: Achieved an accuracy of 74.01% ± 11.34% in cross-subject single-session leave-one-subject-out validation.
- DEAP Dataset: Achieved 61.44% ± 07.15% for valence and 62.86% ± 10.58% for arousal.
- Cross-subject Cross-session: Achieved 80.78% ± 06.12% (three-class) on SEED and 69.55% ± 08.14% (four-class) on SEED-IV, outperforming existing methods with smaller standard deviations, indicating excellent stability.
- Ablation Studies: Confirmed the effectiveness of each component, with the full SDA-DDA model (EXP6) achieving the highest accuracy.
Limitations and Future Work
- Pseudo-label Quality Optimization: While the dynamic confidence threshold improves pseudo-label quality, there's a potential trade-off: excessively high thresholds might discard too many uncertain pseudo-labels, leading to category imbalance. Future work could focus on optimizing the balance between quantity and quality of pseudo-labels, possibly through adaptive thresholds or other robust filtering techniques.
- Real-time Deployment: The paper suggests exploring real-time deployment of SDA-DDA within an online Brain-Computer Interface system. This would allow for continuous model updates based on real-time EEG data, facilitating personalized emotion recognition systems that adapt dynamically.
- Hyperparameter Sensitivity: While the confidence threshold mechanism showed robustness, further investigation into optimal hyperparameter ranges for different scenarios could be beneficial.
Summary updated