Abstract
BrainOmni is the first foundation model to unify Electroencephalography (EEG) and Magnetoencephalography (MEG) signals into a single framework. By addressing device heterogeneity through a physics-grounded BrainTokenizer, the model encodes sensor properties—such as 3D coordinates and types—rather than arbitrary channel names. Pretrained on nearly 2,000 hours of EEG and 650 hours of MEG data, BrainOmni achieves state-of-the-art performance across 11 diverse neurological tasks, demonstrating superior generalization and significant cross-modality synergy.
Introduction
EEG and MEG are non-invasive techniques used to measure brain activity. Despite sharing a common biophysical origin in neural currents, machine learning models for these signals have historically been siloed. The primary obstacle is device heterogeneity: recording devices vary significantly in sensor count, spatial layout, and naming conventions. MEG sensors further complicate matters by measuring different physical quantities (gradients vs. amplitudes) in varying orientations. BrainOmni overcomes these barriers by grounding signal processing in the physical reality of the sensors.
Methods: Unifying the Feature Space
To handle disparate data sources, BrainOmni employs a two-stage training pipeline centered on the BrainTokenizer.
BrainTokenizer and Sensor Encoder
Instead of relying on inconsistent channel names, the Sensor Encoder captures the actual physical properties of each sensor: its 3D Cartesian coordinates, orientation, and type (EEG, MEG gradiometer, or MEG magnetometer). A cross-attention mechanism fuses this spatial data with temporal signal features, mapping external scalp readings to a fixed set of "latent source variables." This architecture mirrors classical "source current estimation" in neuroscience, acting as a data-driven inverse operator. Finally, these variables are quantized into discrete tokens using Residual Vector Quantization (RVQ).
BrainOmni Transformer
Once signals are converted into a unified vocabulary, a Criss-Cross Transformer is trained using a masked token prediction objective. This allows the model to learn rich, spatiotemporal semantic representations of brain activity across both modalities.
Results and Evaluation
BrainOmni was pretrained on a curated dataset of 1,997 hours of EEG and 656 hours of MEG data. It was then evaluated on 11 downstream tasks covering clinical diagnostics (e.g., Alzheimer’s, Parkinson’s, Autism) and functional tasks (e.g., emotion recognition, motor imagery).
State-of-the-Art Performance
BrainOmni consistently outperformed specialized task-specific models and existing EEG foundation models. It achieved approximately 10% higher balanced accuracy on Alzheimer's (AD65) and Parkinson's (PD31) datasets compared to the best pretrained baselines.
Cross-Modality Synergy
A key finding is that joint pretraining on both EEG and MEG (EMEG) improves performance for both modalities. As shown in the table below, BrainOmni achieved a massive 20% improvement on the joint EMEG SomatoMotor task and significantly outperformed baselines on MEG-only clinical tasks like ASD74.
Zero-Shot Generalization
The model demonstrated remarkable zero-shot capabilities, successfully reconstructing signals from entirely unseen EEG and MEG device systems. This confirms that the tokenizer learns universal physical representations rather than memorizing specific hardware configurations.
Conclusion
BrainOmni represents a significant step toward a universal neuro-AI system. By prioritizing physical sensor properties over nomenclature, it provides a scalable framework for learning from diverse neural data. Future work will focus on expanding the pretraining scale and incorporating additional modalities like intracranial EEG (iEEG).
Summary updated