Your Vision Foundation Model Is Secretly an EEG Foundation Model

Guoan Wang, Feng LiuarXiv 2026 · arXiv

TL;DR

The paper introduces SeeEEG, a method that uses a completely frozen vision foundation model (like DINOv3) to encode EEG signals. By simply learning a lightweight "display" module to render brainwaves as images, this approach outperforms specialized EEG foundation models trained on tens of thousands of hours of data across a wide range of clinical and brain-computer interface (BCI) tasks.

What Problem Does This Paper Address?

In recent years, researchers have attempted to build "foundation models" for electroencephalography (EEG) by pretraining large transformers on massive datasets of brain activity. The hope was that these models would learn universal features of brain signals. However, these specialized models have hit a wall: their frozen features often transfer poorly to new tasks, and simply adding more data or parameters hasn't consistently solved the problem.

The authors of this paper take a radical alternative path. They note that neurologists don't "read" raw digital numbers; they interpret EEG by looking at visual traces on a page, identifying patterns like spikes, spindles, and artifacts. If humans use their visual system to process EEG, why shouldn't we use a model already expert at vision?

Methods

The core of the paper is SeeEEG, a framework that treats a Vision Transformer (ViT) as a "black box" EEG encoder.

  1. Frozen Backbone: The vision model (e.g., DINOv3 ViT-B) remains entirely frozen. No EEG data is used to fine-tune its internal weights.
  2. Learned Display: Instead of changing the model, SeeEEG learns how to "show" the data to the model. It optimizes a "display" module that controls:
  3. Montage: How different electrode signals are combined (e.g., subtracting neighboring channels to highlight local activity).
  4. Filtering & Contrast: Adjusting the signal's frequency and visual intensity.
  5. Token-Local Rendering: EEG channels are rendered into a grid of image patches. Each channel occupies a dedicated row of patches, drawn either as a raw waveform ("value view") or a "scalogram" (frequency over time). This ensures that task-relevant patterns fall within the local "tokens" the ViT is designed to process.
  6. Lightweight Optimization: The entire display module contains at most 1,113 trainable parameters—orders of magnitude fewer than the millions of parameters in standard EEG models.
Figure 2. Overview of SeeEEG. (a) The learned display sets the montage, display filter, and contrast, starting from one of the clinical montages that it contains as special cases. (b) Each channel fills a dedicated row of 16-pixel tokens in the value or scalogram view, and long windows are wrapped into a near-square image. (c) A frozen pretrained ViT extracts the class token and the mean patch token of each transformer block. (d) A linear probe or a token MLP reads these tokens, and the readout head and display are selected on validation for each task.

Datasets

The authors evaluated SeeEEG against seven state-of-the-art EEG foundation models (including BENDR, BIOT, and LaBraM) across eight diverse datasets:

  • Clinical: TUEV (event types), TUAB (abnormal detection), ADFTD (dementia), TUSL (slowing).
  • Sleep: HMC (sleep staging).
  • Affective/BCI: SEED (emotion), BCIC2020-3 (imagined speech), PhysioMI (motor imagery).

Key Insights and Ideas

  • The Platonic Representation Hypothesis: The success of SeeEEG supports the idea that models trained on different modalities (images, text, etc.) are converging toward a shared, "Platonic" representation of the world. Because EEG patterns have visual structure, a model trained on billions of natural images already possesses the "visual priors" needed to understand them.
  • Visual Priors > EEG Pretraining: Surprisingly, a model that has never seen a brainwave is a better encoder than one trained specifically on EEG. This suggests that the "features" of vision (edges, textures, temporal dynamics) are more robust and transferable than the features currently learned by specialized EEG models.
  • Input Optimization is Powerful: By focusing on the "lens" (the display) rather than the "brain" (the model), SeeEEG achieves state-of-the-art results with minimal training.
Figure 1. Learning the display through a frozen vision foundation model. EEG foundation models, pretrained on up to tens of thousands of hours of recordings, typically require fine-tuning for each task because their frozen features fall short (left). Neurologists read EEG by visual inspection, and vision foundation models are pretrained on billions of images (middle). SeeEEG shares one frozen ViT backbone across tasks and trains only a lightweight display and readout head for each task (right).

Key Results

  • Dominant Performance: SeeEEG with a frozen DINOv3 backbone outperformed every single EEG foundation model in balanced accuracy on all eight datasets.
  • Mean Rank: It achieved a mean rank of 1.3 across tasks, significantly better than the strongest baseline, REVE, which ranked 2.9.
  • Massive Gains: On specific tasks like "slowing classification" (TUSL), SeeEEG improved the Cohen’s kappa score by +27.1 over the best EEG-specific model.
  • Backbone Agnostic: The learned displays are transferable. When the authors swapped DINOv3 for a video-pretrained model (V-JEPA 2.1), it still outperformed the EEG baselines on six of the eight datasets.
  • The "Why": Ablation studies showed that using a randomly initialized ViT (no pretraining) dropped performance by an average of 15.2 points, proving that the model's "intelligence" comes directly from its visual pretraining on natural images.

Limitations and Future Work

While SeeEEG is highly effective, the authors acknowledge some areas for further exploration:

  • Computational Overhead: Rendering signals into high-resolution images for a ViT can be more computationally intensive than using simple 1D convolutions, which might impact real-time deployment on low-power devices.
  • Beyond EEG: The authors suggest this principle could easily extend to other multichannel physiological signals that experts interpret visually, such as Magnetoencephalography (MEG) or Electrocardiography (ECG).
  • Future Vision Models: As vision foundation models continue to improve, SeeEEG will likely get "smarter" for free, simply by swapping in newer backbones.

Summary updated

Abstract

Electroencephalography (EEG) foundation models are pretrained on thousands of hours of recordings, yet their frozen features transfer poorly to downstream tasks, and this transfer does not consistently improve with larger models or more pretraining data. We show that a vision foundation model that has never seen EEG is a stronger frozen EEG encoder than these specialized models, once the signals are rendered as images that a vision transformer can read. Our method, SeeEEG, keeps the vision backbone frozen and learns only how EEG is displayed to it. Each channel occupies a dedicated row of patches and is drawn either as a waveform at a fixed gain or as a scalogram whose frequency range spans the patch height, so that task-relevant patterns fall within individual tokens. The montage, filtering, and contrast parameters of this display are initialized from a standard clinical montage and optimized with the task loss through the frozen backbone, which amounts to at most 1,113 trainable parameters per view. We compare SeeEEG with seven EEG foundation models on eight clinical, sleep, affective, and brain-computer interface datasets, with every encoder frozen and paired with identical readout heads. With a frozen DINOv3 ViT-B, SeeEEG outperforms every EEG foundation model in balanced accuracy on all eight datasets and reaches a mean rank of 1.3 in Cohen kappa or AUROC, compared with 2.9 for the strongest baseline, REVE. The learned displays transfer without further optimization to the video encoder V-JEPA 2.1, which outperforms every EEG foundation model on both metrics on six of the eight datasets. Replacing DINOv3 with a randomly initialized ViT-B lowers the score by 15.2 points on average, so the ability to read rendered EEG comes from visual pretraining.
BibTeX
@misc{wang2026your,
  author = {Guoan Wang and Feng Liu},
  title = {{Your Vision Foundation Model Is Secretly an EEG Foundation Model}},
  year = {2026},
  eprint = {2611.17031},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17031},
}
Figure