M3D: Manifold-Based Domain Adaptation With Dynamic Distribution for Non-Deep Transfer Learning in Cross-Subject and Cross-Session EEG-Based Emotion Recognition

Ting Luo, Jing Zhang, Yingwei Qiu, Li Zhang, Yaohua Hu, Zhuliang Yu, Zhen LiangIEEE JBHI 2025 · DOI

TL;DR

This paper introduces M3D, a novel non-deep transfer learning framework for EEG-based emotion recognition. M3D addresses the challenges of individual variability and session differences in EEG data by leveraging manifold learning and dynamic distribution alignment, achieving performance comparable to deep learning methods with significantly lower computational demands.

What Problem Does This Paper Address?

Emotion recognition using Electroencephalography (EEG) is a promising area for affective computing, but it faces significant hurdles. EEG signals are inherently non-stationary and vary greatly between individuals and even within the same individual across different sessions. This variability makes it difficult to train models that generalize well. While deep learning approaches have shown promise, they require substantial computational resources and large labeled datasets, which are often scarce in EEG research. This paper aims to bridge this gap by proposing a lightweight, non-deep transfer learning framework that can effectively adapt to these variations without excessive data or computational costs.

Methods

M3D is built upon four key modules:

  • Manifold Feature Transformation: EEG data is first transformed into a lower-dimensional feature space using Transfer Component Analysis (TCA). This reduced-dimension data is then embedded into an optimal Grassmann manifold space. This manifold representation captures the intrinsic geometric properties of the data, making it more robust and suitable for domain adaptation.
  • Dynamic Distribution Alignment: This module addresses the distribution differences between source and target domains. It dynamically adjusts the importance of aligning marginal and conditional distributions based on their significance. An adaptive factor ($\mu$) is introduced, which prioritizes marginal distribution alignment when domains are dissimilar and conditional distribution alignment when they are more similar. This is achieved through an iterative process that refines pseudo-labels for the target domain.
  • Classifier Learning: A classifier is trained on the aligned features. Initially, a weak classifier is trained on the source data. This classifier is then iteratively refined using Structural Risk Minimization (SRM) and the dynamic distribution alignment to minimize distributional differences.
  • Ensemble Learning: To further enhance robustness and prediction accuracy, an ensemble learning approach (LinkCluE) is employed. This aggregates classifiers obtained at different stages of the optimization process, leveraging their diversity.

Datasets

The proposed M3D framework was evaluated on three well-known EEG emotion recognition databases:

  • SEED: Contains EEG data from 15 participants, with three emotions (positive, negative, neutral).
  • SEED-IV: Includes EEG data from 15 participants, eliciting four emotions (happy, sad, neutral, fear).
  • SEED-V: Features EEG data from 16 participants, eliciting five emotions (happiness, sadness, fear, disgust, neutral).

Additionally, the model was validated on a clinical EEG dataset of Major Depressive Disorder (MDD) from 56 participants, demonstrating its applicability in real-world clinical settings.

Key Insights and Ideas

  • Manifold Learning for Robustness: Embedding EEG data into a Grassmann manifold space proves crucial for capturing intrinsic data properties and enhancing domain adaptation, outperforming simpler dimensionality reduction techniques like PCA.
  • Dynamic Adaptation is Key: The adaptive weighting of marginal and conditional distribution alignment ($\mu$) is more effective than fixed weighting, as it dynamically responds to the varying degrees of domain dissimilarity.
  • Non-Deep is Viable: The research demonstrates that a well-designed non-deep transfer learning framework can achieve performance comparable to complex deep learning models, offering a more practical solution for resource-constrained scenarios.
  • Ensemble Learning Boosts Performance: Aggregating classifiers from different learning stages significantly improves the overall robustness and accuracy of the emotion recognition system.

Key Results

  • M3D achieved an average accuracy improvement of 6.67% compared to traditional non-deep learning methods.
  • The framework demonstrated performance comparable to existing deep learning-based methods across various datasets and protocols.
  • On the SEED database, M3D achieved an accuracy of 84.57% under cross-subject single-session validation.
  • For the MDD dataset, M3D achieved accuracies of 82.72% (EC session) and 78.63% (EO session) in cross-subject single-session 10-fold cross-validation, outperforming deep learning methods like DCORAL.
  • The model showed strong robustness to variations in key parameters and achieved high accuracy even with a limited number of training samples.

Limitations and Future Work

  • The authors acknowledge that the adaptive factor $\mu$ was not constrained during the iteration process, which could potentially lead to suboptimal feature alignment in some cases. Future work could address this limitation to further enhance stability and generalizability.
  • The study suggests investigating the adaptability of M3D to more diverse clinical EEG datasets (e.g., epilepsy monitoring, sleep staging) and evaluating its performance under varying preprocessing pipelines.
  • Exploring alternative manifold spaces beyond the Grassmann manifold could also be a promising direction.

Summary updated

BibTeX
@article{luo2025m,
  author = {Ting Luo and Jing Zhang and Yingwei Qiu and Li Zhang and Yaohua Hu and Zhuliang Yu and Zhen Liang},
  title = {{M3D: Manifold-Based Domain Adaptation With Dynamic Distribution for Non-Deep Transfer Learning in Cross-Subject and Cross-Session EEG-Based Emotion Recognition}},
  journal = {IEEE JBHI},
  year = {2025},
  doi = {10.1109/jbhi.2025.3580612},
}
Figure