Brain Structure-Function Fusing Representation Learning using Adversarial Decomposed-VAE for Analyzing MCI

Huisi Wu, Qiankun Zuo, Baiying Lei, Ning Zhong, Yi Pan, Shuqiang WangIEEE Transactions on Neural Systems and Rehabilitation Engineering 2023

TL;DR

This paper introduces the Brain Structure-Function Fusing Representation Learning (BSFL) model, a novel approach that integrates structural (DTI) and functional (fMRI) brain data to better analyze Mild Cognitive Impairment (MCI). By decomposing features into "uniform" and "unique" components and using a knowledge-aware transformer, BSFL effectively fuses these modalities to create unified brain networks for improved MCI prediction and understanding of abnormal brain connections.

What Problem Does This Paper Address?

Mild Cognitive Impairment (MCI) is a critical precursor to Alzheimer's Disease, and early detection is crucial for intervention. While brain imaging techniques like Diffusion Tensor Imaging (DTI) for structural connectivity and functional Magnetic Resonance Imaging (fMRI) for functional connectivity offer valuable insights, effectively combining these two distinct data types to capture the complex brain network abnormalities associated with MCI has been a significant challenge. Previous methods often struggle with accurately fusing these modalities, leading to potential loss of information or reliance on empirical, error-prone feature extraction. This paper aims to bridge this gap by developing a robust framework that can learn a unified representation from both structural and functional data for more accurate MCI diagnosis and analysis.

Methods

The core of the proposed BSFL model lies in its decomposition-fusion framework. This framework involves several key components:

  • Knowledge-Aware Transformer (KAT): This module leverages DTI data to extract structural features. It incorporates prior knowledge about brain regions (e.g., location, volume) from anatomical templates to guide the feature extraction process, aiming to capture both local and global connectivity patterns.
  • Decomposed Variational Graph Autoencoders (Decomposed VAEs): These VAEs are central to the fusion process. They take the extracted structural and functional features and decompose them into two distinct spaces:
  • Uniform Space: Represents common information shared between modalities.
  • Unique Space: Captures modality-specific, complementary information. This decomposition is further guided by a uniform-unique contrastive loss, which encourages distinct representations within each modality while promoting similarity between the uniform representations of different modalities.

  • Representation-Fusing Generator: This component adaptively fuses the decomposed "uniform" and "unique" representations from both structural and functional data. It learns to weigh these components to generate a unified brain network.

  • Dual Discriminator: This consists of two parts:
  • Conventional Discriminator: Ensures the generated brain networks are consistent with real brain network distributions.
  • Label-Aware Discriminator: Classifies the generated networks to distinguish between normal controls and MCI patients, directly contributing to the diagnostic task.

The entire BSFL model is trained end-to-end using a hybrid loss function that combines KL divergence loss, reconstruction loss, adversarial loss, classification loss, and the uniform-unique contrastive loss.

Datasets

The study utilizes data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. This dataset includes subjects with both DTI and fMRI scans, categorized into four groups: Normal Control (NC), Significant Memory Concern (SMC), Early Mild Cognitive Impairment (EMCI), and Late Mild Cognitive Impairment (LMCI). A total of 324 subjects were selected. The paper does not introduce a new benchmark dataset but uses this established one for evaluation.

Key Insights and Ideas

  • Decomposition is Key: The paper highlights that simply concatenating or averaging features from different modalities is insufficient. Decomposing features into "uniform" (common) and "unique" (complementary) components allows for a more nuanced and effective fusion.
  • Prior Knowledge Integration: Incorporating anatomical prior knowledge into the transformer module for structural feature extraction significantly enhances its ability to capture relevant brain network information.
  • Topological Representation: The model moves beyond Euclidean space representations to analyze brain networks in a topological manner, which is crucial for understanding connectivity patterns.
  • Predicting Abnormal Connections: Beyond classification, the BSFL model can identify and visualize specific brain connections that are significantly altered in MCI patients compared to controls, offering potential biomarkers for disease progression.

Key Results

The BSFL model demonstrated superior performance in predicting MCI across different stages compared to several baseline methods (Empirical, GBDM) when evaluated using SVM, DNN, and GCN classifiers. For instance, in the LMCI vs. NC prediction task, the BSFL model achieved the highest accuracy (ACC), sensitivity (SEN), specificity (SPE), and AUC across all classifiers. The paper also shows that the model can effectively reconstruct empirical structural connectivity and functional time series, with low Mean Absolute Errors (MAE) of 0.028 and 0.025 for structural and 0.008 and 0.006 for functional time series in representative subjects. Furthermore, the visualization of abnormal connections revealed patterns consistent with existing clinical findings, identifying both reduced and increased connections that change with disease progression.

Limitations and Future Work

The authors acknowledge two primary limitations:

  • Coarse Brain Region Definition: The use of the Automated Anatomical Labeling (AAL) template for defining brain regions is considered a bit coarse. This might limit the detection of subtle changes within smaller brain regions.
  • Dataset Size: The study used a relatively small dataset (324 subjects).

For future work, the authors suggest:

  • Finer ROI-based Templates: Exploring more detailed anatomical templates to capture finer brain region changes.
  • Larger Datasets: Testing the model on a much larger dataset to further validate its effectiveness and robustness.
  • Parameter Optimization: Investigating ways to control the number of parameters in the model.

Summary updated

Abstract

Integrating the brain structural and functional connectivity features is of great significance in both exploring brain science and analyzing cognitive impairment clinically. However, it remains a challenge to effectively fuse structural and functional features in exploring the complex brain network. In this paper, a novel brain structure-function fusing-representation learning (BSFL) model is proposed to effectively learn fused representation from diffusion tensor imaging (DTI) and resting-state functional magnetic resonance imaging (fMRI) for mild cognitive impairment (MCI) analysis. Specifically, the decomposition-fusion framework is developed to first decompose the feature space into the union of the uniform and unique spaces for each modality, and then adaptively fuse the decomposed features to learn MCI-related representation. Moreover, a knowledge-aware transformer module is designed to automatically capture local and global connectivity features throughout the brain. Also, a uniform-unique contrastive loss is further devised to make the decomposition more effective and enhance the complementarity of structural and functional features. The extensive experiments demonstrate that the proposed model achieves better performance than other competitive methods in predicting and analyzing MCI. More importantly, the proposed model could be a potential tool for reconstructing unified brain networks and predicting abnormal connections during the degenerative processes in MCI.
BibTeX
@misc{wu2023brain,
  author = {Huisi Wu and Qiankun Zuo and Baiying Lei and Ning Zhong and Yi Pan and Shuqiang Wang},
  title = {{Brain Structure-Function Fusing Representation Learning using Adversarial Decomposed-VAE for Analyzing MCI}},
  year = {2023},
  eprint = {2305.14404},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.14404},
}
Figure