Abstract
Increasing evidence has suggested that the dynamic properties of functional brain networks are related to individual behaviors and cognition traits. However, current fMRI-based approaches mostly focus on statistical characteristics of the windowed correlation time course, potentially overlooking subtle time-varying patterns in dynamic functional connectivity (dFC). Here, we proposed the use of an end-to-end deep learning model that combines the convolutional neural network (CNN) and long short-term memory (LSTM) network to capture temporal and spatial features of functional connectivity sequences simultaneously. The results on a large cohort (Human Connectome Project, n = 1,050) demonstrated that our model could achieve a high classification accuracy of about 93% in a gender classification task and prediction accuracies of 0.31 and 0.49 (Pearson's correlation coefficient) in fluid and crystallized intelligence prediction tasks, significantly outperforming previously reported models. Furthermore, we demonstrated that our model could effectively learn spatiotemporal dynamics underlying dFC with high statistical significance based on the null hypothesis estimated using surrogate data. Overall, this study suggests the advantages of a deep learning model in making full use of dynamic information in resting-state functional connectivity, and highlights the potential of time-varying connectivity patterns in improving the prediction of individualized characterization of demographics and cognition traits.
Introduction
Functional Magnetic Resonance Imaging (fMRI) allows researchers to observe the "communication" between various brain regions. Traditionally, studies have focused on Static Functional Connectivity (sFC), which averages brain activity over the entire duration of a scan. However, the brain is inherently dynamic; its networks are constantly reconfiguring to support different cognitive states.
While recent research has moved toward Dynamic Functional Connectivity (dFC) using sliding window techniques, most approaches still rely on simple summary statistics (like the mean or variance of a connection's strength). This "folds" the temporal dimension, potentially discarding the most valuable information: the specific sequence and pattern of how these connections fluctuate. The challenge lies in building a model that can handle the high-dimensional, noisy nature of brain data while simultaneously learning both spatial and temporal features.
Methodology
To address these challenges, we developed an end-to-end deep learning architecture that combines two powerful neural network types:
- Multi-Scale 1D Convolutional Neural Networks (CNN): Instead of a single filter, the model uses three different filter sizes (4, 8, and 16 time windows). This allows the network to extract spatial features of brain connectivity at different "speeds" or scales of brain activity.
- Stacked Long Short-Term Memory (LSTM) Layers: LSTMs are designed to process sequences. Here, they analyze the features extracted by the CNN to identify "short-term sequential switching" and non-linear patterns in how brain states evolve over time.
- Deconvolutional Visualization: To avoid the "black box" problem, deconvolutional layers were used to map the model's high-weight features back onto the brain, identifying which specific connections were driving the predictions.
The study utilized the Human Connectome Project (HCP) S1200 release, consisting of resting-state fMRI (rs-fMRI) scans from 1,050 healthy young adults. Behavioral metrics included fluid intelligence (problem-solving) and crystallized intelligence (accumulated knowledge).
Key Results and Insights
The CNN + LSTM model consistently outperformed traditional machine learning methods like Support Vector Machines (SVM) and Regressions (SVR):
- Gender Classification: Achieved 93.05% accuracy, compared to approximately 87% for traditional models.
- Fluid Intelligence: Achieved a Pearson’s correlation of $r = 0.31$ (outperforming the previous $r = 0.22$).
- Crystallized Intelligence: Achieved a correlation of $r = 0.49$ (outperforming the previous $r = 0.39$).
Crucially, the researchers used "surrogate data" (shuffled data that preserves static correlations but destroys temporal order) to validate the model. The model performed significantly worse on the shuffled data, proving that it was actually learning from the timing of brain fluctuations, rather than just average strengths.
Feature visualization revealed that for gender classification, the most discriminative features were found in the Default Mode Network (DMN) and motor networks. For intelligence prediction, the frontoparietal network—often associated with executive function and complex reasoning—carried the most weight.
Conclusion and Future Work
This study highlights the potential of deep learning to make full use of dynamic information in resting-state functional connectivity. By capturing the "chronnectome," the model provides a more nuanced understanding of individualized demographics and cognition traits. Future work may involve combining fMRI with EEG for higher temporal resolution and applying this architecture to clinical populations to identify dynamic biomarkers for disorders like schizophrenia or autism.
Summary updated