Supervised Brain Network Learning Based on Deep Recurrent Neural Networks

Shijie Zhao, Yan Cui, Linwei Huang, Li Xie, Yaowu Chen, Junwei Han, Lei Guo, Shu Zhang, Tianming Liu, Jinglei LvIEEE Access 2020 · OpenReview

Abstract

Task-based functional magnetic resonance imaging (tfMRI) is a widely used neuroimaging technique for exploring brain networks and functions associated with cognitive behaviors. Traditional model-driven methods, such as the General Linear Model (GLM), often rely on a fixed, non-adaptive Hemodynamic Response Function (HRF), which may overlook diverse and concurrent brain networks. We propose a novel hybrid framework, Supervised brain network learning based on Deep Recurrent Neural Networks (SUDRNN), to reconstruct these complex functional architectures. SUDRNN leverages the capacity of Deep Recurrent Neural Networks (DRNN) to learn diverse regressors from real tfMRI data and utilizes Supervised Dictionary Learning (SDL) to simultaneously reconstruct task-related and latent brain networks. Extensive experiments on Human Connectome Project (HCP) datasets demonstrate the framework's superiority in capturing adaptive, time-delayed, and concurrent brain activity.

Introduction

Understanding how the brain processes information during specific tasks—such as motor movement, emotional recognition, or social interaction—is a fundamental goal of neuroscience. Task-based fMRI (tfMRI) is the primary tool for this exploration. For decades, the General Linear Model (GLM) has been the "gold standard" for analysis. However, GLM's reliance on a predefined, theoretical Hemodynamic Response Function (HRF) presents a significant limitation: it assumes a uniform response across different brain regions and individuals. In reality, brain responses are highly diverse and often delayed. By imposing a rigid model, researchers risk missing "latent" or concurrent networks that do not fit the canonical HRF mold.

To address these limitations, we introduce SUDRNN, a data-driven framework that learns response patterns directly from the signal, allowing for the discovery of previously overlooked functional networks.

The SUDRNN Framework

The SUDRNN framework integrates temporal modeling with spatial decomposition in a two-stage process.

The SUDRNN framework: (a) Task Design Stimuli, (b) DRNN and FC Layers for learning adaptive regressors, (c) Whole Brain Signals, (d) Learned temporal regressors, and (e) Supervised Dictionary Learning for spatial network reconstruction.

Temporal Learning via DRNN

The first stage utilizes Deep Recurrent Neural Networks (DRNNs), specifically Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU) architectures. These models are uniquely suited for fMRI data due to their ability to capture long-term dependencies in sequential data. Unlike basic RNNs, LSTMs and GRUs can "remember" information over time, which is essential for extracting smooth and meaningful brain response curves (regressors) that reflect the actual timing and shape of neural activity.

Spatial Reconstruction via SDL

The learned temporal regressors are then fed into a Supervised Dictionary Learning (SDL) algorithm. This stage treats the whole-brain signal as a combination of various "atoms" in a dictionary. The SDL process simultaneously reconstructs:

  1. Task-Related Networks: Networks directly associated with the experimental stimuli.
  2. Latent Networks: Background or "resting-state" networks (RSNs) that occur concurrently with the task but are often ignored by traditional models.

Experimental Results

The framework was validated using the Human Connectome Project (HCP) 900 subjects release, focusing on 822 subjects across Motor, Emotion, Relational, and Social tasks.

High Fidelity and Sensitivity

SUDRNN's spatial maps for standard motor tasks (e.g., hand or tongue movement) showed high consistency with traditional GLM results, confirming its accuracy. Furthermore, the model successfully identified nine consistent RSNs across different tasks, with spatial overlap rates against established templates ranging from 0.80 to 0.91.

Temporal Flexibility and Robustness

A key insight from the SUDRNN approach is the discovery of multiple time-delayed regressors. The model can track how a network's peak activation shifts over several seconds, providing a dynamic view of brain function. Additionally, the model demonstrated high robustness; training loss curves were nearly identical across independent split-half datasets, indicating that the learned patterns are generalizable across populations.

Conclusion

SUDRNN represents a significant advancement in tfMRI analysis by moving away from rigid, model-driven assumptions toward a flexible, data-driven architecture. By capturing the diverse temporal dynamics of the brain, SUDRNN provides a more holistic view of functional organization. Future research will explore the automation of parameter tuning via Neural Architecture Search (NAS) and the application of this framework to clinical datasets to identify biomarkers for neurological disorders.

Summary updated

Abstract

Task-based functional magnetic resonance imaging (tfMRI) is a widely used neuroimaging technique in exploring brain networks and functions associated with cognitive behaviors. Traditionally, the general linear model (GLM) is the most popular method in tfMRI data analysis due to its simpleness and robustness. This model-driven method adopts a canonical hemodynamic response function (HRF) and its various derivatives to construct regressors in the design matrix and estimate changes in the tfMRI data. However, a possible limitation of current model-driven methods is that the HRF is fixed and non-adaptive which may overlook other diverse and concurrent brain networks. In order to overcome these limitations, we proposed a novel hybrid framework, supervised brain network learning based on deep recurrent neural networks (SUDRNN), to reconstruct the diverse and concurrent functional brain networks. Specifically, this hybrid framework first takes advantage of the great capacity of deep recurrent neural networks (DRNN) in modeling sequential data to learn the diverse regressors from real tfMRI data. After that, it utilizes the effective supervised dictionary learning (SDL) method to reconstruct both the task-related functional brain networks and other latent brain networks simultaneously. Extensive experiment results on different tfMRI datasets from Human connectome project (HCP) demonstrated the superiority of the proposed framework.
BibTeX
@inproceedings{zhao2020supervised,
  author = {Shijie Zhao and Yan Cui and Linwei Huang and Li Xie and Yaowu Chen and Junwei Han and Lei Guo and Shu Zhang and Tianming Liu and Jinglei Lv},
  title = {{Supervised Brain Network Learning Based on Deep Recurrent Neural Networks}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=kOWfYTxaOHN},
}
Figure