Extracting Universal Representations of Cognition across Brain-Imaging Studies

Arthur Mensch, Julien Mairal, Bertrand Thirion, Gaël Varoquaux

Introduction

Functional Magnetic Resonance Imaging (fMRI) research is currently grappling with a significant reproducibility crisis, largely attributed to low statistical power. Most individual studies involve small cohorts (typically a few dozen subjects), making it difficult to distinguish genuine neural signals from background noise. While large-scale data repositories offer a potential solution, aggregating data across diverse studies is challenging due to the lack of standardized experimental paradigms and cognitive ontologies. This paper presents a multi-task learning framework that overcomes these hurdles by learning shared, low-dimensional representations of brain activity, termed Multi-study Task-Optimized Networks (MSTONs).

Methodology: The Multi-Layer Linear Model

The researchers developed a three-layer deep linear model designed to pool information across heterogeneous fMRI datasets without requiring a rigid, pre-defined cognitive theory. The architecture consists of three distinct stages:

  1. Resting-State Prior (Layer 1): To reduce dimensionality and denoise the data, high-dimensional voxel-wise brain images (approx. 200,000 voxels) are projected onto 465 functional units. These units are derived from 4TB of resting-state data using sparse non-negative matrix factorization.
  2. Cognitive Subspace (Layer 2): The model further compresses these 465 units into 128 shared features. This layer is trained jointly across all 35 studies in the corpus, allowing the model to identify common neural patterns associated with various cognitive tasks. This "transfer learning" step is central to the framework's ability to boost predictive power.
  3. Study-Specific Classification (Layer 3): Finally, study-specific "heads" map the 128 shared features back to the original experimental labels (e.g., "face vs. house").
Overview of the multi-layer linear model architecture and data aggregation process.

Crucially, the model remains linear. This design choice ensures that the learned MSTONs are interpretable, allowing researchers to visualize the specific brain networks driving cognitive predictions.

Data and Results

The framework was evaluated using a massive corpus of 35 task-fMRI studies, encompassing 40,000 subject-level contrast maps, 545 unique cognitive contrasts, and over 2,300 subjects. The results demonstrate that the multi-study approach significantly outperforms standard decoding methods.

Decoding performance improvements across 35 studies and analysis of gains relative to study size.

Key findings include:

  • Broad Performance Gains: The multi-study model improved decoding accuracy for 28 out of the 35 studies (80%), with a mean improvement of 5.8% over voxel-based or resting-state network decoders.
  • Impact on Small Studies: The most substantial gains were observed in studies with small sample sizes (e.g., fewer than 30 subjects). By pooling data, these underpowered studies effectively "borrow" statistical strength from the larger aggregate dataset.
  • Functional Atlas Properties: The 128 learned MSTONs function as a new type of functional atlas. Unlike traditional atlases based on anatomy, MSTONs are explicitly optimized to represent cognitive manipulations, mapping onto well-known systems like the language and motor networks with high precision.

Conclusion

By treating diverse fMRI studies as separate tasks within a unified multi-task framework, this research demonstrates that universal representations of cognition can be extracted from heterogeneous data. The resulting MSTONs provide a "universal prior" for brain decoding, significantly enhancing the reliability of small-scale neuroimaging research. The researchers have released the cogspaces library and pre-trained models to facilitate the adoption of these task-optimized networks in the broader neuroscience community.

Summary updated

BibTeX
@article{mensch2021extracting,
  author = {Arthur Mensch and Julien Mairal and Bertrand Thirion and Gaël Varoquaux},
  title = {{Extracting Universal Representations of Cognition across Brain-Imaging Studies}},
  journal = {PLOS Computational Biology},
  year = {2021},
  doi = {10.1371/journal.pcbi.1008795},
}
Figure