Abstract
Decoding emotional states from Electroencephalography (EEG) signals remains a significant challenge due to the non-stationary nature of brain signals and the inherent variability in emotional intensity. This paper presents the STT-CL framework, which integrates a dual-attention Spatial-Temporal Transformer with an intensity-aware curriculum learning strategy. By jointly modeling complex brain patterns and progressively training on samples of increasing difficulty—moving from high-intensity to low-intensity emotions—the model achieves state-of-the-art performance across the SEED, SEED-IV, and DEAP benchmarks.
Introduction
Electroencephalography (EEG) provides a direct window into the human brain's emotional processing, yet translating these high-dimensional, noisy signals into reliable emotion classifications is notoriously difficult. Existing research often struggles with two primary hurdles: the intricate spatiotemporal dependencies of neural oscillations and the "intensity problem," where subtle or low-intensity emotions produce signals that are easily masked by noise.
To address these issues, we introduce the Spatial-Temporal Transformer with Curriculum Learning (STT-CL). This framework treats emotion recognition not just as a classification task, but as a structured learning process that mirrors human development—mastering clear, prototypical emotional expressions before attempting to decode ambiguous, low-intensity fluctuations.
The STT-CL Framework
1. Spatial-Temporal Transformer Architecture
The STT-CL architecture moves beyond generic sequence models by employing a specialized two-stage encoding process designed for the unique topology of EEG data.
- Spatial Encoder: Utilizing multi-head self-attention, the spatial encoder identifies functional connectivity patterns across various EEG channels. Unlike traditional methods that rely on fixed neurophysiological templates, this approach allows the model to dynamically discover which brain regions are most relevant to specific emotional states.
- Temporal Encoder: To capture the transient nature of emotional responses, the model employs a windowed attention mechanism. By applying a sliding window, the temporal encoder prioritizes local dependencies and high-frequency oscillations while maintaining the capacity to model long-range interactions over the entire trial duration.
2. Intensity-Aware Curriculum Learning
The core innovation of this framework is the application of Curriculum Learning (CL) to the problem of emotional intensity. Recognizing that high-intensity emotions (e.g., intense joy or fear) are easier to classify than mild ones, the training process is structured into stages:
- Difficulty Evaluation: Each EEG sample is assigned a difficulty score based on a combination of current prediction loss and historical error rates. High-intensity samples typically yield lower loss and are categorized as "easy."
- Dynamic Scheduling: The model uses an adaptive Gaussian kernel to manage the training distribution. It begins by focusing on a narrow subset of easy samples and gradually expands the training set to include more difficult, low-intensity samples as the model's competence increases.
Experimental Results
The STT-CL framework was validated on three gold-standard datasets, consistently outperforming previous state-of-the-art (SOTA) methods:
| Dataset | Accuracy | Improvement over SOTA |
|---|---|---|
| SEED | 83.84% | +2.21% (vs. FBCCNN) |
| SEED-IV | 76.52% | +1.88% (vs. GCBNet) |
| DEAP | 52.99% | +1.78% (vs. DGCNN) |
Beyond raw accuracy, t-SNE visualizations of the learned feature space indicate that STT-CL produces significantly more distinct clusters with reduced intra-class variation. This suggests that the curriculum learning strategy effectively helps the model find more robust representations that are less sensitive to the noise inherent in low-intensity emotional states.
Discussion and Key Insights
Intensity as a Natural Curriculum
One of the most significant findings is that emotional intensity serves as an effective proxy for sample "hardness." By operationalizing intensity as a curriculum, the model bridges the gap between controlled laboratory data and the subtle neurodynamics of real-world emotional experiences.
Component Synergy
Ablation studies highlighted a critical dependency: the temporal encoder benefits more from the curriculum learning strategy than the spatial encoder. This suggests that temporal dynamics are particularly sensitive to the ambiguity of low-intensity emotions. The synergy between the Transformer's ability to model complex dependencies and the CL's ability to stabilize training allows the model to build strong "priors" on clear signals before tackling noisier data.
Conclusion and Future Work
STT-CL represents a significant step forward in making EEG-based emotion recognition more robust. However, challenges remain in cross-subject generalization, as individual differences in brain anatomy and function still limit the model's ability to transfer knowledge between users. Future research will focus on reducing the computational overhead of the difficulty evaluation process and adapting the curriculum learning strategy for real-time, online applications in wearable BCI devices.
Summary updated