Abstract
Multi-modal neuroimaging technology has significantly enhanced diagnostic accuracy by providing complementary information for identifying objective disease biomarkers. However, conventional deep learning methods often overlook the complex topological relationships between brain regions. While Graph Neural Networks (GNNs) have shown promise in modeling brain connectomes, most existing methods require pre-defined, static graph structures that fail to capture the sophisticated, dynamic nature of the brain. In this study, we propose the Multi-modal Dynamic Graph Convolution Network (MDGCN), a framework designed to learn structural and functional brain networks simultaneously. By utilizing a compositional correspondence matrix and a bilateral graph convolution layer, MDGCN models inter-modal dependencies and aggregates multi-modal representations effectively. Our experiments across three datasets demonstrate state-of-the-art performance, achieving accuracies of 90.4% for Mild Cognitive Impairment (MCI), 85.9% for Parkinson's Disease (PD), and 98.3% for Schizophrenia (SCHZ).
Introduction
The human brain is a complex system of interacting regions, traditionally studied through two distinct lenses: functional connectivity (via fMRI), which tracks temporal correlations in blood flow, and structural connectivity (via DTI), which maps the physical white matter pathways. Integrating these modalities is crucial for a holistic understanding of neurological disorders.
The primary challenge in current neuroimaging research is the "integration gap." Most existing GNNs rely on static connectivity maps that assume a linear or fixed relationship between brain regions. However, the brain's structural-functional relationship is non-linear and dynamic. Furthermore, fusing these heterogeneous data types without losing the nuanced, region-specific interactions remains a significant hurdle for standard machine learning models.
Methodology
The proposed MDGCN framework moves beyond static graph architectures by introducing a dynamic approach to multi-modal learning. The architecture is built upon three core innovations:
1. GRU-based Node Embedding
To move away from static point representations, MDGCN employs Gated Recurrent Units (GRUs) to model brain regions as sequences. This approach captures the sequential relationships within the data and provides a gating mechanism that protects the model from noisy updates, ensuring more robust feature extraction from both functional and structural domains.
2. The Correspondence Matrix and Sinkhorn Normalization
A central feature of MDGCN is the learned correspondence matrix ($\Phi$), which quantifies the "goodness of match" between nodes in the functional and structural domains. To ensure this mapping is mathematically robust and stable, the model applies a Sinkhorn function, making the matrix doubly stochastic. This allows the network to dynamically discover associations between modalities rather than relying on rigid, pre-defined assumptions.
3. Bilateral Graph Convolution (BGC)
Standard GCNs typically operate on a single graph structure. In contrast, the BGC layer aggregates information from both the structural and functional fields simultaneously. By utilizing the correspondence matrix, the BGC layer allows for "cross-pollination" of features, where structural data informs the functional analysis and vice versa, leading to a more comprehensive representation of the brain's state.
Experimental Results
We evaluated MDGCN on three diverse, real-world datasets to test its generalizability across different pathologies and imaging protocols:
- ADNI (Mild Cognitive Impairment): The model achieved 90.4% accuracy, significantly outperforming the best baseline model (84.4%).
- Xuanwu Hospital (Parkinson’s Disease): The model reached 85.9% accuracy in identifying PD patients versus healthy controls.
- CHUV (Schizophrenia): The model achieved a near-perfect 98.3% accuracy.
In all cases, MDGCN consistently outperformed traditional machine learning approaches (SVM, MLP) and specialized brain-network models like BrainNetCNN and M-GCN. The results underscore that the bilateral flow of information between structural and functional graphs is the primary driver of performance gains.
Discussion and Biological Interpretability
Beyond diagnostic accuracy, MDGCN offers biological interpretability. By analyzing the learned correspondence matrix, we found that the model naturally prioritizes brain regions known to be biomarkers for specific diseases. For instance, the model focused heavily on the posterior cingulate gyrus in MCI cases and the somatomotor network in Parkinson's cases, aligning with established clinical evidence.
However, the study also identified a limitation common to GNNs: over-smoothing. As the number of bilateral graph convolution layers increases, the model's performance eventually declines, as excessive global aggregation can wash out the subtle, local structural or functional lesions necessary for accurate diagnosis.
Conclusion
The Multi-modal Dynamic Graph Convolution Network represents a significant step forward in coupling structural and functional connectomes. By learning graph structures dynamically and allowing for bilateral information exchange, MDGCN provides a powerful tool for the automated diagnosis of complex neurological disorders. Future work will explore the integration of additional modalities, such as PET or EEG, to further refine the model's diagnostic capabilities.
Summary updated