LEREL: Lipschitz Continuity-Constrained Emotion Recognition Ensemble Learning For Electroencephalography

Shengyu Gong, Yueyang Li, Zijian Kang, Weiming Zeng, Hongjie Yan, Wai Ting Siok, Nizhuan WangarXiv 2025 · DOI · arXiv

Introduction

Electroencephalography (EEG) serves as a primary modality for objective emotion recognition due to its ability to capture brain activity in real-time. However, EEG signals are characterized by a low Signal-to-Noise Ratio (SNR) and are frequently contaminated by physiological artifacts such as ocular movements or muscular activity.

Existing deep learning architectures for EEG Emotion Recognition (EER) often exhibit significant instability; minor input perturbations can lead to disproportionately large variations in output predictions. Additionally, high intra-subject variability—where neural patterns for the same individual fluctuate over time—hinders model generalization. To address these challenges, the LEREL (Lipschitz Continuity-Constrained Ensemble Learning) framework introduces a mathematically grounded approach to stabilize EER by enforcing Lipschitz constraints across the network.

Methods

The fundamental innovation of LEREL is the systematic application of Lipschitz Continuity. A Lipschitz-constrained function possesses a bounded rate of change, ensuring that small changes in the input result in only proportional, bounded changes in the output. This property is enforced across four specialized branches:

  1. LGCBE (Band Extraction): This module decomposes EEG signals into five standard frequency bands (Delta, Theta, Alpha, Beta, Gamma) using Fast Fourier Transform (FFT), applying Lipschitz constraints to spectral and channel weights.
  2. LGCA (Lipschitz-Constrained Attention): A modified Transformer attention mechanism where attention scores are clamped to a specific range based on a target Lipschitz constant, preventing gradient explosion and over-reliance on noisy outliers.
  3. LGCN (Lipschitz-Constrained Normalization): An evolution of LayerNorm that explicitly bounds the Lipschitz constant of the affine transformation, serving as a stabilizing bottleneck.
  4. Heterogeneous Ensemble Fusion: LEREL integrates four distinct branches—Global Feature, Channel Energy, Channel Feature, and Band Magnitude. These are combined using a learnable weighting strategy ($\alpha$) to dynamically balance the contributions of each expert branch.
The LEREL framework architecture, illustrating the overall ensemble structure (a) and the detailed components: LGCBE (b), LGCN (c), and LGCA (d).

Datasets and Evaluation

The LEREL framework was validated on three prominent public benchmarks:

  • EAV Dataset: 42 participants engaged in natural conversations, categorized into 5 emotion classes.
  • FACED Dataset: A large-scale dataset featuring 123 participants viewing 28 emotion-eliciting videos across 9 categories.
  • SEED Dataset: 15 participants watching film clips to elicit 3 affective states (Positive, Negative, Neutral).

Key Insights

  • Stability via Composition: By ensuring every module (attention, normalization, extraction) is Lipschitz-constrained, the entire deep network inherits a global stability bound.
  • Robustness through Clamping: In the LGCA module, clipping attention scores prevents the model from "obsessing" over noisy outliers, acting as a powerful regularizer for EEG data.
  • Feature Space Regularization: t-SNE visualizations demonstrate that smaller Lipschitz constants lead to more distinct and tighter emotion clusters. Without these constraints, feature representations often overlap significantly, complicating classification.

Key Results

LEREL demonstrated superior performance compared to traditional methods (SVM, KNN) and specialized deep learning models (EEGNet, FBCNet):

  • EAV: Achieved 74.25% accuracy, significantly exceeding the previous benchmark of 59.51%.
  • FACED: Reached 81.19% accuracy, a substantial improvement over existing models (~63%).
  • SEED: Attained 86.79% accuracy.

These results were statistically significant ($p < 0.001$ on FACED), confirming that the performance gains are attributable to the structural robustness of the Lipschitz-constrained framework.

Conclusion

While LEREL effectively handles session drift in intra-subject recognition, cross-subject recognition remains a significant challenge due to individual anatomical and physiological differences. Furthermore, the Lipschitz constant ($K$) is a critical hyperparameter requiring manual tuning; values that are too small may impede learning, while excessively large values can reintroduce instability.

Future research will focus on integrating domain adaptation techniques for cross-subject scenarios and developing adaptive mechanisms to automatically adjust Lipschitz bounds based on real-time environmental noise levels.

Summary updated

BibTeX
@misc{gong2025lerel,
  author = {Shengyu Gong and Yueyang Li and Zijian Kang and Weiming Zeng and Hongjie Yan and Wai Ting Siok and Nizhuan Wang},
  title = {{LEREL: Lipschitz Continuity-Constrained Emotion Recognition Ensemble Learning For Electroencephalography}},
  year = {2025},
  eprint = {2504.09156},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.09156},
}
Figure