A state-space model for inferring effective connectivity of latent neural dynamics from simultaneous EEG/fMRI

Tao Tu, John W. Paisley, Stefan Haufe, Paul SajdaNeurIPS 2019 · OpenReview

Understanding how different brain regions communicate—known as effective connectivity—is a fundamental goal in neuroscience. However, our primary non-invasive tools have inherent limitations. EEG offers excellent temporal resolution (milliseconds) but suffers from "blurry" spatial resolution. Conversely, fMRI provides excellent spatial resolution (millimeters) but is temporally sluggish (seconds).

Historically, researchers have had to choose one modality, leading to a frustrating trade-off. Relying solely on EEG makes it difficult to pinpoint the exact origin of a signal, while fMRI misses the rapid-fire neural interactions that define cognition. To address this, researchers have developed a novel linear state-space model that mathematically integrates simultaneous EEG and fMRI data. By using fMRI to anchor spatial locations and EEG to capture fast-paced dynamics, this model reveals how specific tasks dynamically rewire the brain's communication network.

A Unified State-Space Model

The proposed approach treats neural activity as a hidden (latent) variable that can only be observed indirectly. The methodology bridges the gap between spatial and temporal domains through a multi-step linear state-space model:

  1. Spatial Anchoring via fMRI: The model uses fMRI data to identify subject-specific Regions of Interest (ROIs). These ROIs act as rigid spatial constraints, grounding the source of the signals.
  2. Latent Dynamics: The neural activity at these ROIs is modeled as a Multivariate Autoregressive (MVAR) process. This is a "bilinear" model, meaning external inputs (like viewing a picture) can either directly excite a region or modulate the connection strength between two regions.
  3. Observation Model: A biophysically informed "forward model" describes how these hidden neural states project onto the scalp to create the measurable EEG signals.
  4. Bayesian Inference: To solve the resulting complex mathematical problem, the researchers employ a mean-field variational Bayesian (VB) approach. They also incorporate Automatic Relevance Determination (ARD), which encourages "sparsity"—essentially training the model to identify the most critical connections and filter out noise.
Overview of the linear state-space model integrating fMRI spatial ROIs with EEG forward modeling.

Validating the Model: Simulations and Real-World Data

The model was validated using both simulated data and real-world simultaneous EEG-fMRI recordings. In the real-world dataset, 10 subjects performed a "Face-Car-House" visual categorization task. Subjects viewed images for 100ms and had to rapidly categorize them while inside an MRI scanner, wearing an EEG cap.

fMRI as the Spatial Compass

Simulations revealed that accurate spatial localization from fMRI is critical, not just a supplementary bonus. When the researchers simulated an "EEG-only" scenario by slightly blurring the ROI locations, the model's ability to accurately estimate connectivity plummeted. The EEG-fMRI fusion model significantly outperformed the "EEG-only" approach in recovering the true intrinsic and modulatory connectivity matrices.

Dynamic Rewiring and "House" Complexity

Applying the model to the real-world categorization task demonstrated that the brain's connectivity is not static; it dynamically rewires based on the stimulus. The model identified a specific network involving the Fusiform Face Area (FFA), Parahippocampal Place Area (PPA), Anterior Cingulate Cortex (ACC), and Frontal Eye Fields (FEF).

Effective connectivity maps showing dynamic rewiring when subjects viewed faces versus houses.

A surprising finding was that "House" stimuli induced stronger effective connectivity than "Face" stimuli. Specifically, there was a significant increase in directed connectivity from the ACC to the FEF and FFA when subjects viewed houses compared to faces. The authors suggest that in a rapid-fire task, the brain must work harder—linking more regions for cognitive control and decision monitoring—to distinguish the features of a house, whereas facial processing is highly specialized and almost "automatic."

Limitations and Future Directions

While powerful, the model relies on certain simplifications. It assumes "diagonal state noise," meaning it treats noise in one brain region as independent of noise in another, which may not fully reflect biological reality. Additionally, it assumes the orientation of neural "dipoles" (the direction of electrical flow) is fixed and known.

Future work aims to move beyond using fMRI solely for spatial ROI definition. The ultimate goal is to fully integrate the fMRI time series into the state-space model, paving the way for an even more robust, "super-resolution" map of human brain dynamics.

Summary updated

Abstract

Inferring effective connectivity between spatially segregated brain regions is important for understanding human brain dynamics in health and disease. Non-invasive neuroimaging modalities, such as electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), are often used to make measurements and infer connectivity. However most studies do not consider integrating the two modalities even though each is an indirect measure of the latent neural dynamics and each has its own spatial and/or temporal limitations. In this study, we develop a linear state-space model to infer the effective connectivity in a distributed brain network based on simultaneously recorded EEG and fMRI data. Our method first identifies task-dependent and subject-dependent regions of interest (ROI) based on the analysis of fMRI data. Directed influences between the latent neural states at these ROIs are then modeled as a multivariate autogressive (MVAR) process driven by various exogenous inputs. The latent neural dynamics give rise to the observed scalp EEG measurements via a biophysically informed linear EEG forward model. We use a mean-field variational Bayesian approach to infer the posterior distribution of latent states and model parameters. The performance of the model was evaluated on two sets of simulations. Our results emphasize the importance of obtaining accurate spatial localization of ROIs from fMRI. Finally, we applied the model to simultaneously recorded EEG-fMRI data from 10 subjects during a Face-Car-House visual categorization task and compared the change in connectivity induced by different stimulus categories.
BibTeX
@inproceedings{tu2019state,
  author = {Tao Tu and John W. Paisley and Stefan Haufe and Paul Sajda},
  title = {{A state-space model for inferring effective connectivity of latent neural dynamics from simultaneous EEG/fMRI}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=qyvBu6o3BV},
}
Figure