Abstract
Multivariate analysis of fMRI data has benefited substantially from advances in machine learning. Most recently, a range of probabilistic latent variable models applied to fMRI data have been shown to be effective. However, these models often assume independence in either space or time. This paper introduces a unified framework using Matrix-Normal (MN) distributions to simultaneously model spatial and temporal noise, resulting in faster and more accurate algorithms for Representational Similarity Analysis (RSA) and Shared Response Mapping (SRM).
Introduction
Functional Magnetic Resonance Imaging (fMRI) data is notoriously noisy, with correlations spanning both spatial regions and temporal durations. Modern machine learning methods like Representational Similarity Analysis (RSA) and Shared Response Mapping (SRM) are effective but typically compromise by modeling noise in only one dimension, assuming the other is independent. This isolation of methods makes it difficult to share advancements in noise modeling. This paper addresses this by proposing a unified mathematical framework using Matrix-Normal (MN) models to handle spatiotemporal noise across multiple fMRI tasks.
Methods
The core of the proposed approach is the use of Matrix-Normal (MN) distributions, which factor the massive spatiotemporal covariance matrix into two smaller, manageable matrices: one for space and one for time. This Kronecker-separable covariance model allows for efficient computation.
Two standard methods are reformulated within this framework:
- MN-RSA: A matrix-variate version of RSA that marginalizes over regression weights to avoid bias, modeling residual spatial covariance while capturing temporal structure.
- MN-SRM: A matrix-variate version of SRM that marginalizes over subject-specific mapping matrices and replaces rigid orthonormality constraints with a flexible Gaussian prior.
The authors developed a TensorFlow-based toolkit, matnormal, utilizing linear algebra optimizations like the Woodbury lemma for efficient marginal likelihood computation.
Results
The MN framework yields significant improvements in both speed and accuracy. MN-RSA demonstrated a 10x speed increase over Bayesian RSA on large-scale problems. Notably, at low signal-to-noise ratios (SNR = 0.08), MN-RSA improved Root Mean Square Error (RMSE) by up to 6x compared to baselines.
A critical advantage of MN-RSA is its robustness to false positives. When tested on "null" resting-state data from the Human Connectome Project (HCP), standard RSA methods often hallucinated structured correlations. In contrast, MN-RSA correctly identified a degenerate structure for most subjects, as it estimates temporal and spatial variance simultaneously.
For signal reconstruction, MN-SRM consistently achieved lower out-of-sample reconstruction error than standard SRM and ICA on real-world datasets like Raider and Sherlock. However, the relaxation of constraints in MN-SRM, while aiding reconstruction, slightly reduced performance in downstream linear classification tasks, suggesting that rigid constraints in older methods may serve as effective regularizers for decoding.
Discussion
While the MN framework offers substantial benefits, some limitations remain. MN-RSA slightly underperforms when voxel counts are low and SNR is high, likely due to the assumption of uniform temporal noise across voxels. Future research will explore voxel-specific noise models and deeper integration of SRM and RSA into a unified multi-task learning model for neuroscience.
Summary updated