Spatial-Temporal Transformer with Curriculum Learning for EEG-Based Emotion Recognition

Xuetao Lin, Tianhao Peng, Peihong Dai, Yu Liang, Wenjun WuarXiv 2025 · DOI · arXiv

Abstract

Decoding emotional states from Electroencephalography (EEG) signals remains a significant challenge due to the non-stationary nature of brain signals and the inherent variability in emotional intensity. This paper presents the STT-CL framework, which integrates a dual-attention Spatial-Temporal Transformer with an intensity-aware curriculum learning strategy. By jointly modeling complex brain patterns and progressively training on samples of increasing difficulty—moving from high-intensity to low-intensity emotions—the model achieves state-of-the-art performance across the SEED, SEED-IV, and DEAP benchmarks.

Introduction

Electroencephalography (EEG) provides a direct window into the human brain's emotional processing, yet translating these high-dimensional, noisy signals into reliable emotion classifications is notoriously difficult. Existing research often struggles with two primary hurdles: the intricate spatiotemporal dependencies of neural oscillations and the "intensity problem," where subtle or low-intensity emotions produce signals that are easily masked by noise.

To address these issues, we introduce the Spatial-Temporal Transformer with Curriculum Learning (STT-CL). This framework treats emotion recognition not just as a classification task, but as a structured learning process that mirrors human development—mastering clear, prototypical emotional expressions before attempting to decode ambiguous, low-intensity fluctuations.

The STT-CL Framework

1. Spatial-Temporal Transformer Architecture

The STT-CL architecture moves beyond generic sequence models by employing a specialized two-stage encoding process designed for the unique topology of EEG data.

  • Spatial Encoder: Utilizing multi-head self-attention, the spatial encoder identifies functional connectivity patterns across various EEG channels. Unlike traditional methods that rely on fixed neurophysiological templates, this approach allows the model to dynamically discover which brain regions are most relevant to specific emotional states.
  • Temporal Encoder: To capture the transient nature of emotional responses, the model employs a windowed attention mechanism. By applying a sliding window, the temporal encoder prioritizes local dependencies and high-frequency oscillations while maintaining the capacity to model long-range interactions over the entire trial duration.

2. Intensity-Aware Curriculum Learning

The core innovation of this framework is the application of Curriculum Learning (CL) to the problem of emotional intensity. Recognizing that high-intensity emotions (e.g., intense joy or fear) are easier to classify than mild ones, the training process is structured into stages:

  • Difficulty Evaluation: Each EEG sample is assigned a difficulty score based on a combination of current prediction loss and historical error rates. High-intensity samples typically yield lower loss and are categorized as "easy."
  • Dynamic Scheduling: The model uses an adaptive Gaussian kernel to manage the training distribution. It begins by focusing on a narrow subset of easy samples and gradually expands the training set to include more difficult, low-intensity samples as the model's competence increases.

Experimental Results

The STT-CL framework was validated on three gold-standard datasets, consistently outperforming previous state-of-the-art (SOTA) methods:

Dataset Accuracy Improvement over SOTA
SEED 83.84% +2.21% (vs. FBCCNN)
SEED-IV 76.52% +1.88% (vs. GCBNet)
DEAP 52.99% +1.78% (vs. DGCNN)

Beyond raw accuracy, t-SNE visualizations of the learned feature space indicate that STT-CL produces significantly more distinct clusters with reduced intra-class variation. This suggests that the curriculum learning strategy effectively helps the model find more robust representations that are less sensitive to the noise inherent in low-intensity emotional states.

Discussion and Key Insights

Intensity as a Natural Curriculum

One of the most significant findings is that emotional intensity serves as an effective proxy for sample "hardness." By operationalizing intensity as a curriculum, the model bridges the gap between controlled laboratory data and the subtle neurodynamics of real-world emotional experiences.

Component Synergy

Ablation studies highlighted a critical dependency: the temporal encoder benefits more from the curriculum learning strategy than the spatial encoder. This suggests that temporal dynamics are particularly sensitive to the ambiguity of low-intensity emotions. The synergy between the Transformer's ability to model complex dependencies and the CL's ability to stabilize training allows the model to build strong "priors" on clear signals before tackling noisier data.

Conclusion and Future Work

STT-CL represents a significant step forward in making EEG-based emotion recognition more robust. However, challenges remain in cross-subject generalization, as individual differences in brain anatomy and function still limit the model's ability to transfer knowledge between users. Future research will focus on reducing the computational overhead of the difficulty evaluation process and adapting the curriculum learning strategy for real-time, online applications in wearable BCI devices.

Summary updated

Abstract

EEG-based emotion recognition plays an important role in developing adaptive brain-computer communication systems, yet faces two fundamental challenges in practical implementations: (1) effective integration of non-stationary spatial-temporal neural patterns, (2) robust adaptation to dynamic emotional intensity variations in real-world scenarios. This paper proposes SST-CL, a novel framework integrating spatial-temporal transformers with curriculum learning. Our method introduces two core components: a spatial encoder that models inter-channel relationships and a temporal encoder that captures multi-scale dependencies through windowed attention mechanisms, enabling simultaneous extraction of spatial correlations and temporal dynamics from EEG signals. Complementing this architecture, an intensity-aware curriculum learning strategy progressively guides training from high-intensity to low-intensity emotional states through dynamic sample scheduling based on a dual difficulty assessment. Comprehensive experiments on three benchmark datasets demonstrate state-of-the-art performance across various emotional intensity levels, with ablation studies confirming the necessity of both architectural components and the curriculum learning mechanism.
BibTeX
@misc{lin2025spatial,
  author = {Xuetao Lin and Tianhao Peng and Peihong Dai and Yu Liang and Wenjun Wu},
  title = {{Spatial-Temporal Transformer with Curriculum Learning for EEG-Based Emotion Recognition}},
  year = {2025},
  eprint = {2507.14698},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.14698},
}
Figure