A Simple Review of EEG Foundation Models: Datasets, Advancements and Future Perspectives

Junhong Lai, J. Y. Wei, Yao Lin, Yueming WangarXiv 2025 · DOI · arXiv

TL;DR

This paper provides a comprehensive post-mortem and roadmap for the first generation of EEG Foundation Models (EEG-FMs). By reviewing 14 pioneering models (like Brant-2 and LaBraM), the authors dissect how self-supervised learning is transforming brain-signal analysis from niche, supervised tasks into a scalable, general-purpose field similar to Natural Language Processing.

What Problem Does This Paper Address?

Analyzing Electroencephalogram (EEG) data is notoriously difficult. Signals have a low signal-to-noise ratio, vary wildly between individuals, and are "non-stationary" (they change over time). Traditionally, researchers relied on supervised learning, which requires massive amounts of hand-labeled data—a "bottleneck" because expert medical annotation is expensive and slow.

Furthermore, supervised models often fail to generalize; a model trained to detect seizures in one hospital might fail in another. This paper reviews the shift toward Foundation Models: large-scale architectures pre-trained on vast amounts of unlabeled data to learn the fundamental "language" of the brain before being fine-tuned for specific clinical or research tasks.

Methods

The authors synthesize a "universal framework" for how most current EEG-FMs operate:

  1. Patching: Much like tokens in an LLM, raw EEG time-series are broken into short segments (patches) to reduce computational load and capture local patterns.
  2. Multi-Modal Encoding: Models don't just look at raw voltages. they often fuse Time-Encoding (linear projections), Frequency-Encoding (using Power Spectral Density to capture brain rhythms like Alpha or Beta waves), and Position-Encoding (to track which electrode is where on the scalp).
  3. Transformer Architectures: Most models utilize a dual-encoder setup:
  4. Temporal Encoders: Capture long-range dependencies over time.
  5. Spatial Encoders: Model the interactions between different brain regions (channels).
  6. Self-Supervised Pre-training: The dominant strategy is the Masked Autoencoder (MAE) approach. The model hides a percentage of the EEG patches and tries to reconstruct them. Reconstruction targets vary from raw signal values to "neural codebook" tokens or frequency spectrum features.

Datasets

The review categorizes datasets into two tiers:

  • Pre-training: Massive repositories like the Temple University EEG (TUEG) corpus are standard. The paper notes that Brant-2 currently leads the pack, utilizing a staggering 13.79TB of data (approx. 40,907 hours).
  • Downstream Tasks: Models are evaluated on specialized sets like SEED (emotion recognition), CHB-MIT (seizure detection), and BCIC (motor imagery).
  • The Gap: The authors highlight a lack of standardized benchmarks, noting that different models use different subsets, making "head-to-head" comparisons nearly impossible.

Key Insights and Ideas

  • The Scaling Debate: While LLMs follow strict "scaling laws" (bigger is usually better), EEG-FMs are still finding their footing. The model ALFEE demonstrated that while larger models help, the quality and preprocessing of data are equally critical for performance gains.
  • Frequency Matters: Unlike text or images, EEG has a clear physical interpretation in the frequency domain. Models that explicitly incorporate frequency-band information (like LaBraM) tend to converge better than those relying solely on raw time-series.
  • Spatial "Tokens": Treating each EEG channel as a separate token allows Transformers to learn "brain connectivity" dynamically, rather than relying on fixed, hand-engineered spatial features.
  • The "In-Sample" Trap: A major critique in the paper is that many models are evaluated on "in-sample" data (test data from the same distribution as training). The authors argue for Out-of-Distribution (OOD) testing to prove these models can actually work in real-world clinical settings.

Key Results

  • Model Size: The field has scaled rapidly, from early models with a few million parameters to EEGPT2-Giant at 1.09 Billion parameters.
  • Performance: Foundation models consistently outperform traditional supervised deep learning (like CNNs) on downstream tasks, especially when labeled data for those tasks is scarce (few-shot learning).
  • Hardware: Pre-training these models is resource-intensive; for instance, FoME required 350 hours on 6 NVIDIA RTX 4090s, while Brant-2 used 100 hours on a cluster of A100s.

Limitations and Future Work

  • Standardization: The authors call for a "GLUE" or "ImageNet" equivalent for EEG (mentioning AdaBrain-Bench) to allow for objective model ranking.
  • Long-Term Context: Current models handle "epochs" (seconds), but many brain states (like sleep cycles or cognitive fatigue) evolve over hours. Future models need to bridge this gap.
  • Trustworthy AI: In medicine, "the model said so" isn't enough. There is a dire need for interpretability—tools that show why a foundation model flagged a specific signal as a seizure.
  • Multimodality: The next frontier is "Vertical Fields"—combining EEG with video, eye-tracking, or clinical text to create a holistic view of patient health.

Summary updated

Abstract

Electroencephalogram (EEG) signals play a crucial role in understanding brain activity and diagnosing neurological diseases. Because supervised EEG encoders are unable to learn robust EEG patterns and rely too heavily on expensive signal annotation, research has turned to general-purpose self-supervised EEG encoders, known as EEG-based models (EEG-FMs), to achieve robust and scalable EEG feature extraction. However, the readiness of early EEG-FMs for practical applications and the standards for long-term research progress remain unclear. Therefore, a systematic and comprehensive review of first-generation EEG-FMs is necessary to understand their current state-of-the-art and identify key directions for future EEG-FMs. To this end, this study reviews 14 early EEG-FMs and provides a critical comprehensive analysis of their methodologies, empirical findings, and unaddressed research gaps. This review focuses on the latest developments in EEG-based models (EEG-FMs), which have shown great potential for processing and analyzing EEG data. We discuss various EEG-FMs, including their architectures, pretraining strategies, pretraining and downstream datasets, and other details. This review also highlights challenges and future directions in the field, aiming to provide a comprehensive overview for researchers and practitioners interested in EEG analysis and related EEG-FM.
BibTeX
@misc{lai2025simple,
  author = {Junhong Lai and J. Y. Wei and Yao Lin and Yueming Wang},
  title = {{A Simple Review of EEG Foundation Models: Datasets, Advancements and Future Perspectives}},
  year = {2025},
  eprint = {2504.20069},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.20069},
  howpublished = {arXiv},
}
Figure