Disentangling Spatial-Temporal Functional Brain Networks via Twin-Transformers

Xiaowei Yu, Lu Zhang, Lin Zhao, Yanjun Lyu, Tianming Liu, Dajiang ZhuarXiv 2022 · DOI · arXiv

Introduction

Functional magnetic resonance imaging (fMRI) provides a non-invasive window into neural activity, yet the resulting 4D signals are notoriously complex, with spatial and temporal patterns deeply intertwined. Traditional analysis methods often struggle with two primary challenges: a heavy reliance on prior knowledge of experimental design (e.g., General Linear Models) and a focus on group-average "common" networks that obscure individual-specific functional "fingerprints." To address these limitations, this research introduces a novel "Twin-Transformers" framework designed to disentangle these features using self-supervised learning, enabling the discovery of both universal and unique brain networks without prior task information.

Methodology

The proposed architecture treats fMRI data as a 2D matrix (Time × Space) and employs a dual-transformer approach to process these dimensions concurrently.

Twin-Transformers Architecture

The framework consists of two specialized components:

  1. Spatial Transformer: This module partitions the brain into spatial patches and utilizes self-attention mechanisms to identify voxels belonging to specific functional networks.
  2. Temporal Transformer: This module analyzes time-series data to extract the characteristic signal patterns associated with each identified network.
The Twin-Transformers architecture for disentangling spatial and temporal fMRI features.

Spatial-Temporal Commonality-Variability (ST-CV) Loss

To facilitate learning across subjects, the model incorporates a multi-faceted loss function. It employs Self-Supervised Reconstruction to ensure no information is lost, Spatial Commonality to align maps across subjects, and Temporal Commonality to maximize correlation of shared network patterns. Crucially, the model reserves a subset of network components to capture individual-specific patterns, allowing for the identification of neural signatures that deviate from the group template.

Experimental Setup

The framework was evaluated using the Human Connectome Project (HCP) 3T motor task-fMRI dataset, comprising 917 subjects. The motor task involved specific movements (tongue, fingers, toes) prompted by visual cues. Data was downsampled to 4mm resolution to manage computational overhead, resulting in over 28,000 voxels per subject.

Results and Discussion

The Twin-Transformers model successfully identified task-related and resting-state networks without access to the experimental design.

Task-Free Network Discovery

The model consistently recovered three major task-related networks: Tongue, Right Foot, and Left Foot. The temporal patterns identified for these networks showed a high Pearson Correlation Coefficient (PCC = 0.68) with the actual experimental task design, demonstrating the model's ability to extract meaningful neural structures from raw signals.

Identification of task-related networks (Tongue, Right Foot, Left Foot) across multiple subjects.

Resting-State and Individual Networks

Beyond task-specific activity, the model identified 9 common resting-state networks (e.g., Default Mode Network) active during task performance. Furthermore, it successfully isolated individual-specific networks—regions that synchronize within a single subject but do not align with the group average or task timing. Reproducibility tests yielded Jaccard similarity scores between 0.74 and 0.81, confirming the robustness of the findings.

Conclusion and Future Work

The Twin-Transformers framework represents a significant step toward personalized neuroscience by disentangling the universal and individual components of brain function. Future work will focus on improving computational efficiency to handle higher-resolution data and exploring the clinical utility of individual-specific networks in predicting behavioral traits or diagnosing neurological conditions.

Summary updated

Abstract

How to identify and characterize functional brain networks (BN) is fundamental to gain system-level insights into the mechanisms of brain organizational architecture. Current functional magnetic resonance (fMRI) analysis highly relies on prior knowledge of specific patterns in either spatial (e.g., resting-state network) or temporal (e.g., task stimulus) domain. In addition, most approaches aim to find group-wise common functional networks, individual-specific functional networks have been rarely studied. In this work, we propose a novel Twin-Transformers framework to simultaneously infer common and individual functional networks in both spatial and temporal space, in a self-supervised manner. The first transformer takes space-divided information as input and generates spatial features, while the second transformer takes time-related information as input and outputs temporal features. The spatial and temporal features are further separated into common and individual ones via interactions (weights sharing) and constraints between the two transformers. We applied our TwinTransformers to Human Connectome Project (HCP) motor task-fMRI dataset and identified multiple common brain networks, including both task-related and resting-state networks (e.g., default mode network). Interestingly, we also successfully recovered a set of individual-specific networks that are not related to task stimulus and only exist at the individual level.
BibTeX
@misc{yu2022disentangling,
  author = {Xiaowei Yu and Lu Zhang and Lin Zhao and Yanjun Lyu and Tianming Liu and Dajiang Zhu},
  title = {{Disentangling Spatial-Temporal Functional Brain Networks via Twin-Transformers}},
  year = {2022},
  eprint = {2204.09225},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.09225},
}
Figure