Spatio-Temporal Graph Convolution for Functional MRI Analysis

Soham Gadgil, Qingyu Zhao, Ehsan Adeli, Adolf Pfefferbaum, Edith V. Sullivan, Kilian M. Pohl

Abstract

The Blood-Oxygen-Level-Dependent (BOLD) signal of resting-state fMRI (rs-fMRI) records the temporal dynamics of intrinsic functional networks in the brain. However, existing deep learning methods applied to rs-fMRI either neglect the functional dependency between different brain regions in a network or discard the information in the temporal dynamics of brain activity. To overcome those shortcomings, we propose to formulate functional connectivity networks within the context of spatio-temporal graphs. We train a spatio-temporal graph convolutional network (ST-GCN) on short sub-sequences of the BOLD time series to model the non-stationary nature of functional connectivity. Simultaneously, the model learns the importance of graph edges within ST-GCN to gain insight into the functional connectivities contributing to the prediction. In analyzing the rs-fMRI of the Human Connectome Project (HCP, N=1,091) and the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA, N=773), ST-GCN is significantly more accurate than common approaches in predicting sex and age based on BOLD signals. Furthermore, the brain regions and functional connections significantly contributing to the predictions of our model are important markers according to the neuroscience literature.

Introduction

Resting-state functional MRI (rs-fMRI) measures the Blood-Oxygen-Level-Dependent (BOLD) signal, which serves as a proxy for neural activity while a subject is at rest. Analyzing this data is complex because the brain is a dynamic network where functional relationships often transcend physical proximity. Traditional deep learning approaches typically suffer from two primary limitations: they either collapse temporal data into static correlation matrices, losing dynamic information, or they treat brain regions in isolation using recurrent models like LSTMs, failing to capture the intricate web of inter-regional dependencies. This work addresses these gaps by modeling rs-fMRI data as a spatio-temporal graph, where nodes represent brain regions and edges represent evolving functional connections.

Methods

The proposed framework utilizes Spatio-Temporal Graph Convolutional Networks (ST-GCN) to process BOLD signals. The brain is parcellated into Regions of Interest (ROIs), forming a graph where edges connect the same ROI across consecutive time points (temporal) and different ROIs at the same time point based on functional correlation (spatial).

ST-GCN architecture overview showing sub-sequence training and voting mechanism.

The model employs three layers of spatio-temporal convolutions to extract features jointly across space and time. To enhance interpretability, a Symmetric Edge Importance Matrix ($M$) is introduced, constrained to be positive and symmetric. This allows for the visualization of specific brain connections that contribute most to the model's predictions. Training is performed on short sub-sequences (windows) of BOLD data to account for the non-stationary nature of functional connectivity.

The study validated the model on two large-scale datasets: the NCANDA dataset (N=773 adolescents) for age and sex prediction, and the HCP dataset (N=1,091 young adults) for sex prediction.

Results

The ST-GCN model demonstrated superior performance compared to standard Multi-Layer Perceptrons (MLPs) and LSTMs. On the NCANDA dataset, the model achieved 77.7% accuracy for age prediction and 79.8% for sex prediction. Performance on the young adult HCP dataset reached 83.7% accuracy for sex prediction.

Accuracy of ST-GCN and baseline models across different window sizes for NCANDA Sex and Age prediction.

A key finding was the identification of a "sweet spot" for window size, with optimal performance occurring between 45 and 90 seconds. This aligns with neuroscience literature regarding the dwell time of dynamic functional connectivity. Furthermore, the model's importance maps highlighted neuroscientifically relevant regions, such as the Posterior Cingulate Cortex (PCC)—a central hub of the Default Mode Network—as a primary marker for sex differences.

Discussion and Conclusion

While ST-GCN significantly advances rs-fMRI analysis, several avenues for future research remain. Currently, the spatial graph structure is fixed based on average correlations; future iterations could incorporate dynamic spatial graphs where connections evolve. Additionally, applying the model to finer-grained brain parcellations could yield deeper insights, though at a higher computational cost. Finally, extending this framework to identify biomarkers for neuropsychiatric disorders represents a critical next step for clinical application.

Summary updated

Abstract

The Blood-Oxygen-Level-Dependent (BOLD) signal of resting-state fMRI (rs-fMRI) records the temporal dynamics of intrinsic functional networks in the brain. However, existing deep learning methods applied to rs-fMRI either neglect the functional dependency between different brain regions in a network or discard the information in the temporal dynamics of brain activity. To overcome those shortcomings, we propose to formulate functional connectivity networks within the context of spatio-temporal graphs. We train a spatio-temporal graph convolutional network (ST-GCN) on short sub-sequences of the BOLD time series to model the non-stationary nature of functional connectivity. Simultaneously, the model learns the importance of graph edges within ST-GCN to gain insight into the functional connectivities contributing to the prediction. In analyzing the rs-fMRI of the Human Connectome Project (HCP, N=1,091) and the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA, N=773), ST-GCN is significantly more accurate than common approaches in predicting gender and age based on BOLD signals. Furthermore, the brain regions and functional connections significantly contributing to the predictions of our model are important markers according to the neuroscience literature.
BibTeX
@inproceedings{gadgil2020spatio,
  author = {Soham Gadgil and Qingyu Zhao and Ehsan Adeli and Adolf Pfefferbaum and Edith V. Sullivan and Kilian M. Pohl},
  title = {{Spatio-Temporal Graph Convolution for Functional MRI Analysis}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  doi = {10.1007/978-3-030-59728-3\_52},
}
Figure