DMMR: Cross-Subject Domain Generalization for EEG-Based Emotion Recognition via Denoising Mixed Mutual Reconstruction

Yiming Wang, Bin Zhang, Yujiao TangAAAI 2024 · DOI

Abstract

Electroencephalography (EEG)-based emotion recognition is hindered by significant inter-subject variability, making it difficult for models to generalize to new individuals. We propose Denoising Mixed Mutual Reconstruction (DMMR), a novel framework designed for cross-subject domain generalization. DMMR employs a two-stage pre-training strategy centered on a Mutual Reconstruction (MMR) module that forces the encoder to learn subject-invariant emotional markers. Additionally, a "Time Steps Shuffling" noise injection method is introduced to improve robustness against temporal jitter. Experimental results on the SEED and SEED-IV datasets demonstrate that DMMR achieves state-of-the-art performance, effectively bridging the gap between individual brain signatures.

Introduction

While EEG is a high-fidelity tool for monitoring emotional states, its practical application is limited by the "individual difference" problem. Unique brain structures and subjective emotional experiences result in distinct signal patterns for different people. Traditional Domain Adaptation (DA) methods attempt to mitigate this by requiring data from the target subject during training. However, Domain Generalization (DG)—where the model must perform on a completely unseen subject—remains a significant challenge.

Current DG approaches often struggle with task interference between subject-independence and emotion classification, data scarcity in EEG datasets, and inherent signal noise. DMMR addresses these issues by decoupling feature extraction from classification through a specialized pre-training paradigm.

Methodology

The DMMR framework is built upon several key components designed to extract robust, subject-neutral features.

Mutual Reconstruction (MMR) and Data Augmentation

The core of DMMR is the MMR module. Unlike standard autoencoders, MMR utilizes a multi-decoder architecture. During pre-training, the encoder processes data from one subject, while the decoders attempt to reconstruct features corresponding to other subjects within the same emotional category. This process compels the encoder to discard subject-specific "noise" and focus on universal emotional indicators. To combat data scarcity, DMMR implements a two-stage mixed data augmentation where outputs from multiple decoders are blended to create synthetic "mix-subject" features, enriching the training distribution.

The DMMR framework architecture featuring the Mutual Reconstruction (MMR) module and DG-DANN for subject-invariant feature extraction.

Denoising via Time Steps Shuffling

To enhance model stability, the authors introduce Time Steps Shuffling. This noise injection technique shuffles the temporal segments of an EEG sample while keeping the final time step fixed. Since LSTM-based encoders rely heavily on the final state to represent the current emotion, this method tests the model's ability to handle temporal variations without losing critical emotional context.

Adversarial Training and Attention

The framework also incorporates a Domain-Adversarial Neural Network (DG-DANN) module. A subject discriminator works against the encoder, ensuring that the extracted features are indistinguishable across different individuals. Furthermore, an Attention-Based Pooling (ABP) layer prioritizes emotion-rich frequency bands, such as Beta and Gamma waves, and critical brain channels.

Experimental Results

DMMR was evaluated using a "leave-one-subject-out" cross-validation scheme on the SEED and SEED-IV datasets.

Performance Benchmarks

On the SEED dataset, DMMR achieved a state-of-the-art accuracy of 88.27%, outperforming existing methods like GMSS (86.52%) and PPDA (86.70%). On the SEED-IV dataset, it reached 72.70% accuracy, remaining competitive with current top-performing models. Ablation studies indicated that while mixed data augmentation significantly improves accuracy, the Time Steps Shuffling technique is crucial for reducing performance variance across subjects, thereby increasing model reliability.

Feature Visualization

T-SNE analysis provides clear evidence of the model's efficacy. In the pre-trained stage, features from different subjects show significant overlap, indicating successful domain generalization. Following fine-tuning, these features form distinct clusters based on emotion categories, demonstrating that the model retains high discriminative power.

T-SNE visualizations of subject and emotion distributions across original, pre-trained, and fine-tuned features.

Conclusion

The DMMR framework effectively addresses the challenges of cross-subject EEG emotion recognition. By leveraging mutual reconstruction and strategic noise injection, it achieves superior generalization to unseen individuals. Future work will focus on optimizing these techniques for smaller datasets and exploring real-time applications in wearable health monitoring devices.

Summary updated

BibTeX
@inproceedings{wang2024dmmr,
  author = {Yiming Wang and Bin Zhang and Yujiao Tang},
  title = {{DMMR: Cross-Subject Domain Generalization for EEG-Based Emotion Recognition via Denoising Mixed Mutual Reconstruction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i1.27819},
}
Figure