Beyond Accuracy: Robustness, Interpretability and Expressiveness of EEG Foundation Models

Urban Širca, Maryam Alimardani, Stefanos Zafeiriou, Konstantinos BarmpasarXiv 2026 · arXiv

TL;DR

This paper presents the first systematic evaluation of Electroencephalography (EEG) foundation models (EEG-FMs) beyond simple "clean" accuracy, investigating their robustness to noise, neurophysiological interpretability, and representational quality. By benchmarking six state-of-the-art EEG-FMs across eight datasets, the authors reveal that current models exhibit distinct failure modes under signal corruption, but their underlying representations are surprisingly expressive and spatially aligned with human brain physiology. Crucially, they show that previously reported poor linear probing performance is largely a byproduct of sub-optimal token pooling rather than deficient pre-trained features.

What Problem Does This Paper Address?

While foundation models have revolutionized NLP and computer vision, their adaptation to EEG data is still in its infancy. Current benchmarks for EEG-FMs focus almost exclusively on classification accuracy using clean, curated laboratory data. However, real-world EEG data is notoriously messy, plagued by electrode impedance drifts, subject movement, muscle artifacts, and channel dropouts.

Furthermore, we don't fully understand why these massive models make their predictions, whether they actually learn features that align with known human neurophysiology, or why "head-only" linear probing (evaluating the frozen pre-trained model) consistently underperforms full fine-tuning. This paper addresses these critical gaps in robustness, interpretability, and expressiveness.

Methods

The authors benchmarked six prominent EEG foundation models—BIOT, CBraMod, LaBraM, NeuroRVQ, BrainOmni, and REVE—against a compact, supervised deep learning baseline (EEGNet).

Their evaluation framework introduced three novel layers of analysis:

  1. Robustness to Perturbations: Evaluating models at test-time against additive white/pink noise, random channel dropout, and region-specific signal corruption. They compared zero-padding dropped channels against completely removing them for variable-channel architectures.
  2. Interpretability via AttnLRP: The authors presented the first application of Attention-Aware Layer-Wise Relevance Propagation (AttnLRP) to EEG transformers to map exactly which brain regions the models rely on. For models where AttnLRP was numerically unstable, Gradient×Input (G×I) was utilized.
  3. Expressiveness & Probing: They conducted block-wise linear probing across transformer depths to locate where task-relevant information emerges, alongside an ablation study comparing different token aggregation strategies (mean pooling vs. token flattening).

Datasets

The models were evaluated across eight distinct datasets covering six Brain-Computer Interface (BCI) paradigms:

  • Movement / Movement*: High-Gamma and PhysioNet motor execution datasets (4 classes).
  • Motor-Imagery / Motor-Imagery*: OpenBMI and PhysioNet motor imagery datasets (2 and 4 classes).
  • ERP: OpenBMI visual P300 event-related potentials.
  • Eyes: PhysioNet eyes open vs. eyes closed resting state.
  • Memory: Pavlov digit span working memory task.
  • Sleep: Sleep EDF 6-class sleep staging dataset.

Key Insights and Ideas

  • No Single Model Rules Them All: Robustness profiles are highly architecture-dependent. For example, BrainOmni is incredibly resilient to additive white noise but highly fragile under zero-padded channel dropout. Conversely, REVE handles channel dropout best but degrades faster under global noise.
  • The "Zero-Padding" Trap: The fragility of variable-channel models to channel dropout isn't due to the loss of information, but rather how they process zeros. When dropped channels are completely removed from the input token sequence instead of being zero-padded, performance drops nearly disappear.
  • Physiological Alignment: EEG-FMs are not just exploiting spurious correlations; their internal attention mechanisms naturally focus on task-appropriate brain regions (e.g., motor tasks focus on central sensorimotor strips, while visual ERPs focus on the occipital lobe), as shown in Figure 3.
  • Stable Attention, Corrupted Content: Under heavy noise, a model's spatial attribution maps remain remarkably stable even as its classification accuracy plummets. This implies that the models still "look" at the correct brain regions but are failing to decode the corrupted signal content within those regions.
  • Rethinking Representation Quality: The common assumption that EEG-FMs learn poor general representations because linear probing fails is incorrect. The culprit is actually mean pooling, which collapses crucial spatial and temporal token information. Switching to flatten pooling (concatenating tokens) dramatically recovers performance without updating the backbone weights.

Key Results

  • Clean Performance: Downstream gains of EEG-FMs over supervised baselines remain modest. Under full fine-tuning, the top models clustered closely, with NeuroRVQ leading average balanced accuracy (0.696) against the tiny baseline EEGNet (0.652). BIOT failed to learn across all downstream tasks.
Classification balanced accuracy of full fine-tuned and head-only fine-tuned foundation models and deep learning baseline across 8 downstream tasks
  • Noise Resilience: BrainOmni demonstrated exceptional noise robustness, losing only 5.4% accuracy at -5 dB white noise, compared to a 14.1%–20.9% drop for all other models (Figure 2).
  • Pooling Strategy Fix: Replacing mean pooling with token flattening during head-only probing yielded massive accuracy boosts: +21% for NeuroRVQ and +14% for LaBraM, effectively closing the gap with the supervised baseline.
  • Information Emergence: Block-wise probing revealed that task-relevant information peaks in the early-to-middle blocks of pre-trained models and declines toward the final layers, confirming that later layers specialize heavily toward the self-supervised pre-training proxy task.

Limitations and Future Work

A primary limitation is that all models were evaluated using synthetic test-time perturbations rather than real-world artifacts (like actual eye blinks, muscle movements, or physical electrode displacement). Additionally, all models were originally trained strictly on clean data; future work should explore whether incorporating noise augmentation during pre-training or fine-tuning can bridge the observed robustness gaps. Finally, the attribution analyses were restricted to correctly classified trials, leaving open the question of what features drive the models' mistakes.

Summary updated

Abstract

EEG foundation models (EEG-FMs) have been evaluated predominantly on clean, in-distribution accuracy, leaving their robustness, interpretability and representational quality largely unexamined. This study addresses these gaps by benchmarking six EEG-FMs against a baseline deep learning model across eight datasets. Beyond clean accuracy, we conduct three layers of analysis: (i) Robustness: we apply test-time perturbations including additive noise, random and region-based channel dropout and region-specific noise injection. Our analyses show that no single model dominates all failure modes. The most noise-robust model is among the most fragile under channel dropout and much of the dropout fragility disappears when channels are removed rather than zero-padded. (ii) Interpretability: we present the first application of Attention-Aware Layer-Wise Relevance Propagation (AttnLRP) to EEG-FMs and show that models broadly concentrate relevance on task-appropriate brain regions consistent with known neurophysiology. However, attribution maps remain spatially stable under perturbation while predictions degrade, suggesting that the models attend to the correct brain regions but decode corrupted content. (iii) Expressiveness: With block-wise probing we show that late blocks are repurposed during fine-tuning, while early blocks already hold task-related information. Furthermore, we demonstrate that the poor head-only performance previously attributed to low-quality pre-trained representations is largely explained by pooling and that EEG-FMs possess sufficient representational capacity when their token-level embeddings are preserved. Together, these findings provide the first systematic assessment of robustness, interpretability and expressiveness for EEG-FMs and highlight critical considerations for their development.
BibTeX
@misc{sirca2026beyond,
  author = {Urban Širca and Maryam Alimardani and Stefanos Zafeiriou and Konstantinos Barmpas},
  title = {{Beyond Accuracy: Robustness, Interpretability and Expressiveness of EEG Foundation Models}},
  year = {2026},
  eprint = {2605.17562},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.17562},
}

Loading related content...

Figure