Abstract
We introduce the Brain Language Model (BrainLM), a foundation model for brain activity dynamics trained on 6,700 hours of fMRI recordings. Utilizing self-supervised masked-prediction training, BrainLM demonstrates proficiency in both fine-tuning and zero-shot inference tasks. Fine-tuning allows for the accurate prediction of clinical variables like age, anxiety, and PTSD as well as forecasting of future brain states. Critically, the model generalizes well to entirely new external cohorts not seen during training. In zero-shot inference mode, BrainLM can identify intrinsic functional networks directly from raw fMRI data without any network-based supervision during training. The model also generates interpretable latent representations that reveal relationships between brain activity patterns and cognitive states. Overall, BrainLM offers a versatile and interpretable framework for elucidating the complex spatiotemporal dynamics of human brain activity. It serves as a powerful “lens” through which massive repositories of fMRI data can be analyzed in new ways, enabling more effective interpretation and utilization at scale.
Introduction
Functional magnetic resonance imaging (fMRI) provides a non-invasive window into the living brain by measuring blood-oxygen-level dependent (BOLD) signals. However, this data is notoriously difficult to work with: it is high-dimensional, noisy, and possesses complex spatiotemporal dynamics that vary significantly between individuals.
Traditionally, researchers have built "narrow" machine learning models designed for specific tasks—like identifying a specific disease or decoding a single cognitive state. These models often fail to generalize to new datasets and struggle to leverage the vast repositories of unlabeled fMRI data available today. BrainLM addresses this by creating a flexible, generative framework that learns the underlying "grammar" of brain activity from massive amounts of data without needing explicit labels.
Methods and Architecture
The researchers adapted the Masked Autoencoder (MAE) architecture—a staple in computer vision and natural language processing—for fMRI time series. The core of BrainLM is a Transformer-based model trained via self-supervision.
- Preprocessing: The brain is parcellated into 424 regions (using the AAL-424 atlas), converting 4D fMRI data into 424-dimensional time series.
- Self-Supervised Training: During training, the model is "blinded" to a large portion of the data (up to 75% masking ratio). It must then learn to reconstruct the missing segments by attending to the visible parts of the recording. This forces the model to learn the intricate dependencies of how different brain regions interact over time.
- In Silico Perturbation: A novel "prompting" technique allows the model to act as a simulator. By nudging the model's latent representations, researchers can observe how the predicted brain state changes, effectively performing "virtual experiments."
Datasets and Generalization
The model was trained and evaluated on two of the largest fMRI repositories in existence:
- UK Biobank (UKB): Approximately 42,000 individuals (77,000 recordings, 6,700 hours). This served as the primary training and internal testing set.
- Human Connectome Project (HCP): 1,002 high-quality recordings used as an independent external validation set to prove the model's ability to generalize across different scanners and populations.
Key Insights
- Scaling Matters: Training on 1-2 orders of magnitude more data than previous attempts resulted in more robust and generalizable representations.
- Emergent Functional Organization: The model’s internal self-attention weights naturally aligned with known functional networks (e.g., visual or default mode networks) without any supervision. The model "discovered" the functional organization of the brain on its own.
- Zero-Shot Versatility: BrainLM can perform "zero-shot" inference to identify functional networks or simulate perturbations without any additional training.
- A Foundation for Biomarkers: Fine-tuning the pre-trained model proved significantly more effective than training from scratch on raw data for clinical tasks.
Results
- Generalization: BrainLM achieved an average $R^2$ score of 0.402 on held-out UKB data and 0.316 on the entirely external HCP dataset.
- Clinical Prediction: Fine-tuning reduced the Mean Squared Error (MSE) for age prediction from 2.0 (raw data) to 0.485, significantly outperforming traditional baselines like SVMs.
- Forecasting: BrainLM outperformed specialized architectures like LSTMs and Neural ODEs in predicting future brain states (20 timesteps ahead).
- Functional Identification: The model classified brain regions into functional networks with 58.8% accuracy using internal attention weights, compared to 39.2% using raw data.
Conclusion and Future Work
BrainLM represents a significant shift toward "Big Data" in neuroscience, suggesting that Transformer-based revolutions can help decode the complex spatiotemporal dynamics of the human brain. While successful, future work aims to increase spatial resolution to the voxel level, integrate multi-modal data (EEG, MEG, genomics), and expand testing to a wider array of neurodegenerative and psychiatric conditions.
Summary updated