Abstract
Electroencephalogram (EEG), which directly reflects the emotional activity of the brain, has been increasingly utilized for emotion recognition. Most works exploit the spatial and temporal dependencies in EEG to learn emotional feature representations, but they still have two limitations to reach their full potential. First, prior knowledge is rarely used to capture the spatial dependency of brain regions. Second, the cross temporal dependency between consecutive time slices for different brain regions is ignored. To address these limitations, in this paper, we propose Variational Spatial and Gaussian Temporal (VSGT) graph models to investigate the spatial and temporal dependencies for EEG-based emotion recognition. The VSGT has two key components: Variational Spatial Encoder (VSE) and Gaussian Temporal Encoder (GTE). The VSE leverages the upper bound theorem to identify the dynamic spatial dependency based on prior knowledge by the variational Bayesian method. Besides, the GTE exploits the conditional Gaussian graph transform that computes comprehensive temporal dependency between consecutive time slices. Finally, the VSGT utilizes a recurrent structure to calculate the spatial and temporal dependencies for all time slices. Extensive experiments show the superiority of VSGT over state-of-the-art methods on multiple EEG datasets.
Introduction
Electroencephalogram (EEG) signals are considered a "gold standard" for tracking emotional states due to their direct reflection of brain activity. However, current deep learning models for EEG-based emotion recognition often treat the brain as a "black box," failing to incorporate established neurological principles. This research identifies two critical gaps: the neglect of biological spatial priors and the failure to account for asynchronous "lags" in signal propagation across different brain regions.
The proposed Variational Spatial and Gaussian Temporal (VSGT) model addresses these by integrating physiological prior knowledge with a novel mechanism to track cross-region temporal influences. By combining a Variational Spatial Encoder (VSE) based on neural field theory and a Gaussian Temporal Encoder (GTE) that captures cross-region delays, the model achieves state-of-the-art performance while maintaining biological plausibility.
Methodology
The VSGT architecture is designed to process EEG data as a sequence of graphs, where nodes represent electrodes and edges represent functional dependencies. The model consists of three primary modules:
Variational Spatial Encoder (VSE)
Rather than learning a static adjacency matrix from scratch, the VSE incorporates Neural Field Theory, which posits that brain activity arises from the superposition of traveling waves. It models spatial dependency as a distribution rather than a fixed value. By applying the De Moivre-Laplace Theorem, the model approximates a Binomial distribution of connections with a Gaussian proxy. This allows the use of the "reparameterization trick" to make complex biological priors differentiable and trainable, ensuring the model identifies dynamic spatial patterns grounded in physiology.
Gaussian Temporal Encoder (GTE)
Brain activity is inherently asynchronous; an emotional stimulus in one region may propagate to another with a slight temporal delay. The GTE addresses this "lag" problem using a conditional Gaussian graph transform. This component calculates dependencies between different brain regions across consecutive time slices (e.g., how activity in region A at time $t-1$ influences region B at time $t$), capturing comprehensive temporal dynamics that standard models miss.
Recurrent Spatial and Temporal (RST) Module
These encoders are integrated into a recurrent structure. The RST module processes the sequence of EEG graphs, utilizing a Graph Convolutional Network (GCN) to extract features while maintaining a hidden state "memory" to infer the evolving emotional state over time.
Experimental Results
Performance Benchmarks
The VSGT model was evaluated on two major public datasets: DEAP (32 subjects) and DREAMER (23 subjects). Using 10-fold cross-validation on 4-second segments, VSGT outperformed a wide array of baselines, including non-graph methods (ResLSTM), pure graph methods (DGCNN), and hybrid spatio-temporal models (ASTG-LSTM).
- DEAP Dataset: Achieved 78.95% accuracy for Arousal and 74.15% for Valence.
- DREAMER Dataset: Achieved 75.19% accuracy for Arousal and 73.41% for Valence.
Interpretability and Neurological Alignment
A significant advantage of the VSGT model is its interpretability. Visualization of the learned spatial dependencies reveals patterns that align with established neurological findings. For positive emotions, high correlations are concentrated in the frontal lobe ($Fp1, Fp2$). Conversely, negative emotions exhibit higher correlations in the central sulcus ($C3, C4$). This demonstrates that the model's reliance on biological priors allows it to identify meaningful emotional signatures rather than just statistical noise.
Conclusion
VSGT represents a significant advancement in EEG-based emotion recognition by bridging the gap between deep learning and neuroscience. By constraining the model with physiological priors and accounting for cross-temporal delays, the authors achieve both higher accuracy and greater interpretability.
Future work may address the computational overhead of all-to-all edge embeddings to enable real-time application on wearable devices. Additionally, exploring adaptive time windows and cross-subject generalization remains a priority for making these models viable for broader clinical and consumer use cases.
Summary updated