Multi-modal Dynamic Graph Network: Coupling Structural and Functional Connectome for Disease Diagnosis and Classification

Yanwu Yang, Xutao Guo, Zhikai Chang, Chenfei Ye, Yang Xiang, Ting Ma

TL;DR

Researchers have developed a Multi-modal Dynamic Graph Convolution Network (MDGCN) that effectively "couples" structural and functional brain data to diagnose neurological disorders. By treating brain regions as dynamic nodes that interact across different imaging modalities, the model achieved state-of-the-art accuracy in identifying Mild Cognitive Impairment (90.4%), Parkinson’s (85.9%), and Schizophrenia (98.3%).

What Problem Does This Paper Address?

The human brain is the ultimate complex system. To understand it, clinicians use different "lenses": fMRI (functional MRI) tracks blood flow to see which areas are active together, while DTI (Diffusion Tensor Imaging) maps the physical "wiring" or structural pathways.

The challenge lies in integration. Most existing Graph Neural Networks (GNNs) rely on static, pre-defined connectivity maps. However, brain connectivity is non-linear and sophisticated; a simple linear correlation doesn't capture how structure influences function (and vice versa). Furthermore, most models struggle to fuse these two different types of data (modalities) without losing the nuanced relationships between specific brain regions.

Methods

The authors propose the MDGCN, which moves away from static graphs toward a more fluid, "dynamic" representation. Key innovations include:

  1. GRU-based Node Embedding: Instead of treating brain regions as static points, they use Gated Recurrent Units (GRUs) to model node representations as sequences. This helps capture sequential relationships and protects the model from "noisy" or undesired updates.
  2. The Correspondence Matrix: This is the "secret sauce." The model calculates a soft correspondence matrix ($\Phi$) that measures the "goodness of match" between nodes in the functional and structural domains. They apply a Sinkhorn function to ensure this matrix is doubly stochastic, creating a robust mapping between modalities.
  3. Bilateral Graph Convolution (BGC): Unlike standard GCNs that operate on a single graph, the BGC layer aggregates information from both fields simultaneously. It uses the correspondence matrix to "cross-pollinate" features, allowing the structural data to inform the functional analysis and vice versa.

Datasets

The model was validated on three diverse, real-world datasets:

  • ADNI: Focused on Mild Cognitive Impairment (MCI) vs. Healthy Controls (NC).
  • Xuanwu Hospital Dataset: Focused on Parkinson’s Disease (PD) vs. Healthy Controls.
  • CHUV Dataset: Focused on Schizophrenia (SCHZ) vs. Healthy Controls.

These datasets provided a rigorous test of the model's ability to generalize across different pathologies and imaging protocols.

Key Insights and Ideas

  • Dynamic vs. Static: The biggest "aha" moment is that learning the graph structure is better than assuming it. By letting the model discover the associations between brain regions dynamically, it bypasses the limitations of traditional linear connectivity measures.
  • Cross-Modal Synergy: The research confirms that functional and structural connectomes are deeply intertwined. The model performs significantly better when it can "reason" across both modalities rather than looking at them in isolation.
  • Biological Interpretability: The model isn't just a black box. By analyzing the learned correspondence matrix, the researchers found that the model naturally focused on regions like the posterior cingulate gyrus (a known biomarker for Alzheimer's/MCI) and the somatomotor network (linked to Parkinson's).

Key Results

The MDGCN consistently outperformed traditional machine learning (SVM, MLP) and specialized brain-network models (BrainNetCNN, M-GCN, HGNN).

  • MCI Diagnosis (ADNI): Achieved 90.4% accuracy, a significant jump over the next best baseline (84.4%).
  • Parkinson’s Diagnosis (Xuanwu): Achieved 85.9% accuracy.
  • Schizophrenia Diagnosis (CHUV): Achieved a near-perfect 98.3% accuracy.

The results show that the "bilateral" approach—where information flows between the structural and functional graphs—is the primary driver of these performance gains.

Limitations and Future Work

The authors noted a common pitfall in GNNs: over-smoothing. As they increased the number of bilateral graph convolution layers, performance initially improved but then began to decline. This suggests that too much global aggregation can wash out the local, subtle lesions necessary for diagnosis.

Future work could involve exploring even more modalities (like PET scans or EEG) or refining the dynamic mapping to handle even larger, more complex brain parcellations without succumbing to the over-smoothing effect.

Summary updated

Abstract

Multi-modal neuroimaging technology has greatlly facilitated the efficiency and diagnosis accuracy, which provides complementary information in discovering objective disease biomarkers. Conventional deep learning methods, e.g. convolutional neural networks, overlook relationships between nodes and fail to capture topological properties in graphs. Graph neural networks have been proven to be of great importance in modeling brain connectome networks and relating disease-specific patterns. However, most existing graph methods explicitly require known graph structures, which are not available in the sophisticated brain system. Especially in heterogeneous multi-modal brain networks, there exists a great challenge to model interactions among brain regions in consideration of inter-modal dependencies. In this study, we propose a Multi-modal Dynamic Graph Convolution Network (MDGCN) for structural and functional brain network learning. Our method benefits from modeling inter-modal representations and relating attentive multi-model associations into dynamic graphs with a compositional correspondence matrix. Moreover, a bilateral graph convolution layer is proposed to aggregate multi-modal representations in terms of multi-modal associations. Extensive experiments on three datasets demonstrate the superiority of our proposed method in terms of disease classification, with the accuracy of 90.4%, 85.9% and 98.3% in predicting Mild Cognitive Impairment (MCI), Parkinson's disease (PD), and schizophrenia (SCHZ) respectively. Furthermore, our statistical evaluations on the correspondence matrix exhibit a high correspondence with previous evidence of biomarkers.
BibTeX
@inproceedings{yang2022multi,
  author = {Yanwu Yang and Xutao Guo and Zhikai Chang and Chenfei Ye and Yang Xiang and Ting Ma},
  title = {{Multi-modal Dynamic Graph Network: Coupling Structural and Functional Connectome for Disease Diagnosis and Classification}},
  booktitle = {BIBM},
  year = {2022},
  doi = {10.1109/bibm55620.2022.9995642},
}

Loading related content...

Figure