FEMBA: Efficient and Scalable EEG Analysis with a Bidirectional Mamba Foundation Model

Anna Tegon, Thorir Mar Ingolfsson, Xiaying Wang, Luca Benini, Yawei LiarXiv 2025 · DOI · arXiv

Introduction

Electroencephalography (EEG) is a cornerstone of clinical neurology, essential for diagnosing conditions like epilepsy and monitoring brain health. However, the manual analysis of long-term EEG recordings is labor-intensive and prone to variability. While Transformer-based models have recently achieved state-of-the-art results in automated EEG analysis, their quadratic complexity ($O(N^2)$) relative to sequence length poses significant barriers. This computational burden makes them difficult to deploy on resource-constrained wearable devices or for processing the very long recordings typical in clinical settings.

FEMBA (Foundational EEG Mamba + Bidirectional Architecture) addresses these challenges by utilizing a state-space model (SSM) architecture that scales linearly ($O(N)$). This framework enables high-fidelity brain monitoring with significantly reduced computational overhead compared to traditional attention-based models.

Methodology

The core innovation of FEMBA is the integration of a bidirectional Mamba architecture. Standard Mamba models are causal, meaning they only process information in a forward temporal direction. However, clinical EEG interpretation often requires context from both preceding and succeeding signal segments to identify specific biomarkers, such as interictal spikes. FEMBA employs parallel forward and backward Mamba components to capture this global context effectively.

The FEMBA pipeline: from raw EEG input and masking to the bidirectional encoder and downstream task heads.

The model is trained using a self-supervised masked signal reconstruction task. During pre-training on the Temple University Hospital EEG (TUEG) Corpus—comprising over 21,000 hours of data from more than 5,000 subjects—the model learns to reconstruct 60% of masked EEG patches. This process forces the encoder to develop robust, generalizable representations of brain activity without requiring manual labels. FEMBA is developed in four scales, ranging from Tiny (7.8M parameters) for edge devices to Huge (389M parameters) for high-performance clinical applications.

Experimental Results

FEMBA demonstrates that high accuracy can be maintained alongside massive gains in efficiency. The model was evaluated on several downstream tasks, including abnormal EEG detection (TUAB), artifact detection (TUAR), and slowing event detection (TUSL).

  • State-of-the-Art Performance: On the TUAR artifact detection task, the FEMBA-Tiny model established a new benchmark, outperforming previous state-of-the-art models while being 27x more computationally efficient.
  • Competitive Accuracy: In the TUAB task, FEMBA-Huge achieved 81.82% balanced accuracy, coming within 0.7% of the leading Transformer model (LaBraM) while utilizing 3.5x fewer FLOPs and 1.5x less memory.
Comparison of FLOPs and Peak GPU Memory Usage between LaBraM, FEMBA, and EEGFormer. Note the significant efficiency gains for FEMBA across all sizes.

Discussion and Conclusion

The results suggest that the attention mechanism, while powerful, is not strictly necessary for high-performance EEG analysis. Linear-time architectures like Mamba offer a viable path toward deploying sophisticated foundation models on hardware that fits on a patient's wrist or head, rather than requiring a server rack.

Despite these gains, challenges remain. The model currently faces difficulties with domain shifts, such as transitioning from adult to neonatal EEG data. Future research will focus on multi-modal integration—combining EEG with heart rate or movement data—and improving the interpretability of Mamba’s internal dynamics to provide clinically meaningful explanations for its decisions.

Summary updated

Abstract

Accurate and efficient electroencephalography (EEG) analysis is essential for detecting seizures and artifacts in long-term monitoring, with applications spanning hospital diagnostics to wearable health devices. Robust EEG analytics have the potential to greatly improve patient care. However, traditional deep learning models, especially Transformer-based architectures, are hindered by their quadratic time and memory complexity, making them less suitable for resource-constrained environments. To address these challenges, we present FEMBA (Foundational EEG Mamba + Bidirectional Architecture), a novel self-supervised framework that establishes new efficiency benchmarks for EEG analysis through bidirectional state-space modeling. Unlike Transformer-based models, which incur quadratic time and memory complexity, FEMBA scales linearly with sequence length, enabling more scalable and efficient processing of extended EEG recordings. Trained on over 21,000 hours of unlabeled EEG and fine-tuned on three downstream tasks, FEMBA achieves competitive performance in comparison with transformer models, with significantly lower computational cost. Specifically, it reaches 81.82% balanced accuracy (0.8921 AUROC) on TUAB and 0.949 AUROC on TUAR, while a tiny 7.8M-parameter variant demonstrates viability for resource-constrained devices. These results pave the way for scalable, general-purpose EEG analytics in both clinical and highlight FEMBA as a promising candidate for wearable applications.
BibTeX
@misc{tegon2025femba,
  author = {Anna Tegon and Thorir Mar Ingolfsson and Xiaying Wang and Luca Benini and Yawei Li},
  title = {{FEMBA: Efficient and Scalable EEG Analysis with a Bidirectional Mamba Foundation Model}},
  year = {2025},
  eprint = {2502.06438},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.06438},
}
Figure