TL;DR
The paper introduces SeeEEG, a method that uses a completely frozen vision foundation model (like DINOv3) to encode EEG signals. By simply learning a lightweight "display" module to render brainwaves as images, this approach outperforms specialized EEG foundation models trained on tens of thousands of hours of data across a wide range of clinical and brain-computer interface (BCI) tasks.
What Problem Does This Paper Address?
In recent years, researchers have attempted to build "foundation models" for electroencephalography (EEG) by pretraining large transformers on massive datasets of brain activity. The hope was that these models would learn universal features of brain signals. However, these specialized models have hit a wall: their frozen features often transfer poorly to new tasks, and simply adding more data or parameters hasn't consistently solved the problem.
The authors of this paper take a radical alternative path. They note that neurologists don't "read" raw digital numbers; they interpret EEG by looking at visual traces on a page, identifying patterns like spikes, spindles, and artifacts. If humans use their visual system to process EEG, why shouldn't we use a model already expert at vision?
Methods
The core of the paper is SeeEEG, a framework that treats a Vision Transformer (ViT) as a "black box" EEG encoder.
- Frozen Backbone: The vision model (e.g., DINOv3 ViT-B) remains entirely frozen. No EEG data is used to fine-tune its internal weights.
- Learned Display: Instead of changing the model, SeeEEG learns how to "show" the data to the model. It optimizes a "display" module that controls:
- Montage: How different electrode signals are combined (e.g., subtracting neighboring channels to highlight local activity).
- Filtering & Contrast: Adjusting the signal's frequency and visual intensity.
- Token-Local Rendering: EEG channels are rendered into a grid of image patches. Each channel occupies a dedicated row of patches, drawn either as a raw waveform ("value view") or a "scalogram" (frequency over time). This ensures that task-relevant patterns fall within the local "tokens" the ViT is designed to process.
- Lightweight Optimization: The entire display module contains at most 1,113 trainable parameters—orders of magnitude fewer than the millions of parameters in standard EEG models.
Datasets
The authors evaluated SeeEEG against seven state-of-the-art EEG foundation models (including BENDR, BIOT, and LaBraM) across eight diverse datasets:
- Clinical: TUEV (event types), TUAB (abnormal detection), ADFTD (dementia), TUSL (slowing).
- Sleep: HMC (sleep staging).
- Affective/BCI: SEED (emotion), BCIC2020-3 (imagined speech), PhysioMI (motor imagery).
Key Insights and Ideas
- The Platonic Representation Hypothesis: The success of SeeEEG supports the idea that models trained on different modalities (images, text, etc.) are converging toward a shared, "Platonic" representation of the world. Because EEG patterns have visual structure, a model trained on billions of natural images already possesses the "visual priors" needed to understand them.
- Visual Priors > EEG Pretraining: Surprisingly, a model that has never seen a brainwave is a better encoder than one trained specifically on EEG. This suggests that the "features" of vision (edges, textures, temporal dynamics) are more robust and transferable than the features currently learned by specialized EEG models.
- Input Optimization is Powerful: By focusing on the "lens" (the display) rather than the "brain" (the model), SeeEEG achieves state-of-the-art results with minimal training.
Key Results
- Dominant Performance: SeeEEG with a frozen DINOv3 backbone outperformed every single EEG foundation model in balanced accuracy on all eight datasets.
- Mean Rank: It achieved a mean rank of 1.3 across tasks, significantly better than the strongest baseline, REVE, which ranked 2.9.
- Massive Gains: On specific tasks like "slowing classification" (TUSL), SeeEEG improved the Cohen’s kappa score by +27.1 over the best EEG-specific model.
- Backbone Agnostic: The learned displays are transferable. When the authors swapped DINOv3 for a video-pretrained model (V-JEPA 2.1), it still outperformed the EEG baselines on six of the eight datasets.
- The "Why": Ablation studies showed that using a randomly initialized ViT (no pretraining) dropped performance by an average of 15.2 points, proving that the model's "intelligence" comes directly from its visual pretraining on natural images.
Limitations and Future Work
While SeeEEG is highly effective, the authors acknowledge some areas for further exploration:
- Computational Overhead: Rendering signals into high-resolution images for a ViT can be more computationally intensive than using simple 1D convolutions, which might impact real-time deployment on low-power devices.
- Beyond EEG: The authors suggest this principle could easily extend to other multichannel physiological signals that experts interpret visually, such as Magnetoencephalography (MEG) or Electrocardiography (ECG).
- Future Vision Models: As vision foundation models continue to improve, SeeEEG will likely get "smarter" for free, simply by swapping in newer backbones.
Summary updated