Multi-modal Dynamic Graph Network: Coupling Structural and Functional Connectome for Disease Diagnosis and Classification

Yanwu Yang, Xutao Guo, Zhikai Chang, Chenfei Ye, Yang Xiang, Ting Ma

Abstract

Multi-modal neuroimaging technology has significantly enhanced diagnostic accuracy by providing complementary information for identifying objective disease biomarkers. However, conventional deep learning methods often overlook the complex topological relationships between brain regions. While Graph Neural Networks (GNNs) have shown promise in modeling brain connectomes, most existing methods require pre-defined, static graph structures that fail to capture the sophisticated, dynamic nature of the brain. In this study, we propose the Multi-modal Dynamic Graph Convolution Network (MDGCN), a framework designed to learn structural and functional brain networks simultaneously. By utilizing a compositional correspondence matrix and a bilateral graph convolution layer, MDGCN models inter-modal dependencies and aggregates multi-modal representations effectively. Our experiments across three datasets demonstrate state-of-the-art performance, achieving accuracies of 90.4% for Mild Cognitive Impairment (MCI), 85.9% for Parkinson's Disease (PD), and 98.3% for Schizophrenia (SCHZ).

Introduction

The human brain is a complex system of interacting regions, traditionally studied through two distinct lenses: functional connectivity (via fMRI), which tracks temporal correlations in blood flow, and structural connectivity (via DTI), which maps the physical white matter pathways. Integrating these modalities is crucial for a holistic understanding of neurological disorders.

The primary challenge in current neuroimaging research is the "integration gap." Most existing GNNs rely on static connectivity maps that assume a linear or fixed relationship between brain regions. However, the brain's structural-functional relationship is non-linear and dynamic. Furthermore, fusing these heterogeneous data types without losing the nuanced, region-specific interactions remains a significant hurdle for standard machine learning models.

Methodology

The proposed MDGCN framework moves beyond static graph architectures by introducing a dynamic approach to multi-modal learning. The architecture is built upon three core innovations:

1. GRU-based Node Embedding

To move away from static point representations, MDGCN employs Gated Recurrent Units (GRUs) to model brain regions as sequences. This approach captures the sequential relationships within the data and provides a gating mechanism that protects the model from noisy updates, ensuring more robust feature extraction from both functional and structural domains.

2. The Correspondence Matrix and Sinkhorn Normalization

A central feature of MDGCN is the learned correspondence matrix ($\Phi$), which quantifies the "goodness of match" between nodes in the functional and structural domains. To ensure this mapping is mathematically robust and stable, the model applies a Sinkhorn function, making the matrix doubly stochastic. This allows the network to dynamically discover associations between modalities rather than relying on rigid, pre-defined assumptions.

3. Bilateral Graph Convolution (BGC)

Standard GCNs typically operate on a single graph structure. In contrast, the BGC layer aggregates information from both the structural and functional fields simultaneously. By utilizing the correspondence matrix, the BGC layer allows for "cross-pollination" of features, where structural data informs the functional analysis and vice versa, leading to a more comprehensive representation of the brain's state.

Experimental Results

We evaluated MDGCN on three diverse, real-world datasets to test its generalizability across different pathologies and imaging protocols:

  • ADNI (Mild Cognitive Impairment): The model achieved 90.4% accuracy, significantly outperforming the best baseline model (84.4%).
  • Xuanwu Hospital (Parkinson’s Disease): The model reached 85.9% accuracy in identifying PD patients versus healthy controls.
  • CHUV (Schizophrenia): The model achieved a near-perfect 98.3% accuracy.

In all cases, MDGCN consistently outperformed traditional machine learning approaches (SVM, MLP) and specialized brain-network models like BrainNetCNN and M-GCN. The results underscore that the bilateral flow of information between structural and functional graphs is the primary driver of performance gains.

Discussion and Biological Interpretability

Beyond diagnostic accuracy, MDGCN offers biological interpretability. By analyzing the learned correspondence matrix, we found that the model naturally prioritizes brain regions known to be biomarkers for specific diseases. For instance, the model focused heavily on the posterior cingulate gyrus in MCI cases and the somatomotor network in Parkinson's cases, aligning with established clinical evidence.

However, the study also identified a limitation common to GNNs: over-smoothing. As the number of bilateral graph convolution layers increases, the model's performance eventually declines, as excessive global aggregation can wash out the subtle, local structural or functional lesions necessary for accurate diagnosis.

Conclusion

The Multi-modal Dynamic Graph Convolution Network represents a significant step forward in coupling structural and functional connectomes. By learning graph structures dynamically and allowing for bilateral information exchange, MDGCN provides a powerful tool for the automated diagnosis of complex neurological disorders. Future work will explore the integration of additional modalities, such as PET or EEG, to further refine the model's diagnostic capabilities.

Summary updated

Abstract

Multi-modal neuroimaging technology has greatlly facilitated the efficiency and diagnosis accuracy, which provides complementary information in discovering objective disease biomarkers. Conventional deep learning methods, e.g. convolutional neural networks, overlook relationships between nodes and fail to capture topological properties in graphs. Graph neural networks have been proven to be of great importance in modeling brain connectome networks and relating disease-specific patterns. However, most existing graph methods explicitly require known graph structures, which are not available in the sophisticated brain system. Especially in heterogeneous multi-modal brain networks, there exists a great challenge to model interactions among brain regions in consideration of inter-modal dependencies. In this study, we propose a Multi-modal Dynamic Graph Convolution Network (MDGCN) for structural and functional brain network learning. Our method benefits from modeling inter-modal representations and relating attentive multi-model associations into dynamic graphs with a compositional correspondence matrix. Moreover, a bilateral graph convolution layer is proposed to aggregate multi-modal representations in terms of multi-modal associations. Extensive experiments on three datasets demonstrate the superiority of our proposed method in terms of disease classification, with the accuracy of 90.4%, 85.9% and 98.3% in predicting Mild Cognitive Impairment (MCI), Parkinson's disease (PD), and schizophrenia (SCHZ) respectively. Furthermore, our statistical evaluations on the correspondence matrix exhibit a high correspondence with previous evidence of biomarkers.
BibTeX
@inproceedings{yang2022multi,
  author = {Yanwu Yang and Xutao Guo and Zhikai Chang and Chenfei Ye and Yang Xiang and Ting Ma},
  title = {{Multi-modal Dynamic Graph Network: Coupling Structural and Functional Connectome for Disease Diagnosis and Classification}},
  booktitle = {BIBM},
  year = {2022},
  doi = {10.1109/bibm55620.2022.9995642},
}
Figure