Multi network InfoMax: A pre-training method involving graph convolutional networks

Usman Mahmood, Zening Fu, Vince D. Calhoun, Sergey M. PlisarXiv 2021 · OpenReview

Abstract

Discovering distinct features and their relations from data can help us uncover valuable knowledge crucial for various tasks, e.g., classification. In neuroimaging, these features could help to understand, classify, and possibly prevent brain disorders. Model introspection of highly performant overparameterized deep learning (DL) models could help find these features and relations. However, to achieve high-performance level DL models require numerous labeled training samples ($n$) rarely available in many fields. This paper presents a pre-training method involving graph convolutional/neural networks (GCNs/GNNs), based on maximizing mutual information between two high-level embeddings of an input sample. Many of the recently proposed pre-training methods pre-train one of many possible networks of an architecture. Since almost every DL model is an ensemble of multiple networks, we take our high-level embeddings from two different networks of a model --a convolutional and a graph network--. The learned high-level graph latent representations help increase performance for downstream graph classification tasks and bypass the need for a high number of labeled data samples. We apply our method to a neuroimaging dataset for classifying subjects into healthy control (HC) and schizophrenia (SZ) groups. Our experiments show that the pre-trained model significantly outperforms the non-pre-trained model and requires $50\%$ less data for similar performance.

Introduction

In neuroimaging research, particularly with functional MRI (fMRI), scientists often encounter a "curse of dimensionality." While a single scan provides tens of thousands of measurements, the number of available subjects is typically limited to a few dozen or hundred. Traditional machine learning approaches often rely on hand-crafted features to manage this dimensionality, which can inadvertently discard critical temporal dynamics. While deep learning (DL) models are capable of capturing these complex patterns, they are notoriously data-hungry and prone to failure when labeled data is scarce. This study introduces Multi network InfoMax (MIM), a self-supervised pre-training strategy designed to overcome these limitations by learning robust brain dynamics from large, unlabeled datasets.

Methodology

The MIM framework utilizes an ensemble architecture that processes fMRI Region of Interest (ROI) time series through a 1D-CNN to capture temporal features, followed by a self-attention layer. The resulting output is then processed by a Gated Graph Neural Network (GNN) to model spatial and functional relationships between brain regions.

Architecture of the Multi network InfoMax (MIM) model, showing the integration of CNN and GNN components for pre-training.

The core innovation of MIM is its "multi-network" pre-training approach. The model generates two global embeddings for each subject: one from the convolutional module ($y$) and another from the graph module ($c$). By employing a critic function and the InfoNCE loss, the model maximizes the mutual information between these two embeddings. This process "syncs" the modules, forcing the GNN to incorporate global graph information and the CNN to extract features relevant to the overall brain structure.

For evaluation, the model was pre-trained on the Human Connectome Project (HCP) dataset, which contains unlabeled scans from healthy controls. Downstream training and testing were conducted using the Function Biomedical Informatics Research Network (FBIRN) dataset, consisting of 368 subjects categorized into Healthy Control (HC) and Schizophrenia (SZ) groups.

Results and Discussion

The MIM pre-training strategy demonstrated significant improvements in both performance and data efficiency. The pre-trained model achieved state-of-the-art results, significantly outperforming traditional machine learning methods like SVM and Logistic Regression, which struggled with the raw ROI data format.

Performance comparison (AUC) between MIM and the non-pre-trained BrainGNN across varying training set sizes.

A key finding is the model's data efficiency: the pre-trained MIM model achieved an Area Under the Curve (AUC) comparable to the non-pre-trained baseline (BrainGNN) while requiring 50% less training data. At a full training size of 100 subjects per class, MIM reached approximately 0.83 AUC, compared to 0.78 for the non-pre-trained version. These gains were statistically significant ($p < 0.05$) across nearly all training set sizes.

The success of MIM suggests that the architectural relationship between components can serve as a powerful "teacher." By forcing the CNN and GNN to agree on subject representations, the model learns features that generalize well from healthy populations to clinical classification tasks.

Limitations and Future Work

While effective, the authors note that the current study utilized region-based data rather than raw 4D fMRI data. Applying MIM to full spatio-temporal fMRI data could yield even more profound benefits. Future research may also explore the applicability of this "multi-network syncing" to other hybrid architectures, such as Transformer-GNN ensembles, and test its generalization across other neurological disorders like Alzheimer’s or Autism.

Summary updated

Abstract

Discovering distinct features and their relations from data can help us uncover valuable knowledge crucial for various tasks, e.g., classification. In neuroimaging, these features could help to understand, classify, and possibly prevent brain disorders. Model introspection of highly performant overparameterized deep learning (DL) models could help find these features and relations. However, to achieve high-performance level DL models require numerous labeled training samples ($n$) rarely available in many fields. This paper presents a pre-training method involving graph convolutional/neural networks (GCNs/GNNs), based on maximizing mutual information between two high-level embeddings of an input sample. Many of the recently proposed pre-training methods pre-train one of many possible networks of an architecture. Since almost every DL model is an ensemble of multiple networks, we take our high-level embeddings from two different networks of a model --a convolutional and a graph network--. The learned high-level graph latent representations help increase performance for downstream graph classification tasks and bypass the need for a high number of labeled data samples. We apply our method to a neuroimaging dataset for classifying subjects into healthy control (HC) and schizophrenia (SZ) groups. Our experiments show that the pre-trained model significantly outperforms the non-pre-trained model and requires $50\%$ less data for similar performance.
BibTeX
@misc{mahmood2021multi,
  author = {Usman Mahmood and Zening Fu and Vince D. Calhoun and Sergey M. Plis},
  title = {{Multi network InfoMax: A pre-training method involving graph convolutional networks}},
  year = {2021},
  eprint = {2111.01276},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.01276},
}
Figure