Pre-train Graph Neural Networks for Brain Network Analysis

Yi Yang, Hejie Cui, Carl YangICLR 2023 · OpenReview

TL;DR

This paper introduces PTGB, a novel pre-training framework for Graph Neural Networks (GNNs) designed to analyze brain networks. PTGB addresses the critical challenge of limited labeled data in neuroscience by leveraging unsupervised learning and a unique atlas mapping technique to enable knowledge transfer across diverse brain network datasets.

What Problem Does This Paper Address?

Understanding the human brain, the control center for our cognition and behavior, is a major scientific endeavor. Neuroscientists are increasingly interested in how different brain regions (Regions of Interest or ROIs) interact to influence neural development and diagnose disorders. Graph Neural Networks (GNNs) are powerful tools for analyzing these complex, interconnected brain networks. However, training deep learning models like GNNs typically requires vast amounts of labeled data. In the realm of brain networks, acquiring such large, labeled datasets is particularly challenging due to the complexities of data acquisition and strict privacy regulations, leading to relatively small datasets compared to other domains. This scarcity of data can lead to overfitting and hinder the development of robust models for tasks like disease prediction. Furthermore, different studies often use varying methods to define ROIs (parcellation atlases), making it difficult to directly apply models trained on one dataset to another.

Methods

PTGB tackles these challenges with a two-pronged approach:

  1. Unsupervised Pre-training for Brain Networks: PTGB employs a novel, unsupervised pre-training strategy specifically designed for brain networks. This method learns intrinsic brain network structures without relying on task-specific labels, allowing it to leverage large, publicly available datasets. The core of this pre-training involves a two-level contrastive learning approach. This strategy aims to maximize the mutual information between an anchor node's representation and its positive samples (e.g., its neighbors within a certain hop distance) while minimizing it with negative samples. This is crucial for learning robust representations that capture local and global network properties.
  2. Data-Driven Parcellation Atlas Mapping: To overcome the issue of inconsistent ROI definitions across datasets, PTGB introduces a data-driven atlas mapping pipeline. This pipeline transforms features from different datasets into a common, lower-dimensional space. It uses a linear autoencoder with specialized regularizers (locality-preserving, modularity-aware, and sparsity-oriented) to ensure that the compressed features retain spatial relationships, capture community structures within the brain, and are robust to noise. Crucially, it then employs variance-based sorting to align these virtual ROIs across datasets, ensuring that similar functional modules are represented consistently, regardless of the original parcellation.

The framework also incorporates a MAML (Model-Agnostic Meta-Learning) inspired approach for multi-task pre-training, allowing it to learn a generalizable initialization that can be quickly adapted to various downstream tasks.

Datasets

The study utilizes three real-world brain network datasets:

  • Parkinson's Progression Markers Initiative (PPMI): A large-scale dataset used for pre-training.
  • Bipolar Disorder (BP) dataset: A local dataset used for downstream evaluation.
  • Human Immunodeficiency Virus Infection (HIV) dataset: Another local dataset used for downstream evaluation.

These datasets differ in their parcellation atlases (e.g., Desikan-Killiany84 for PPMI, Brodmann82 for BP, and AAL90 for HIV), highlighting the need for the atlas mapping component of PTGB.

Key Insights and Ideas

  • Unsupervised pre-training is crucial for brain network analysis: The scarcity of labeled data in neuroscience makes unsupervised learning and pre-training essential for building effective GNN models.
  • Contrastive learning can be tailored for brain networks: The proposed two-level contrastive sampling, considering both local neighborhoods and inter-graph relationships, significantly boosts performance.
  • Atlas mapping is a critical enabler for cross-dataset learning: The data-driven atlas mapping pipeline is vital for harmonizing brain network data from different sources, a major hurdle in the field.
  • Local neighborhood information is paramount: Methods that effectively capture k-hop neighborhood information (like those in the EGS category) perform best, underscoring the importance of local brain structure.
  • Multi-task learning enhances generalizability: Pre-training on multiple datasets simultaneously (MTP) leads to better performance and reduced variance compared to single-task pre-training.

Key Results

  • PTGB consistently outperformed all baselines across various GNN architectures (GCN, GAT, GIN) in disease prediction tasks, showing relative improvements of 7.34%-13.30% over the best-performing baselines.
  • The proposed two-level contrastive sampling and atlas mapping regularizers were shown to be critical components, with their removal leading to significant performance drops.
  • The sparsity-oriented regularizer (SR) was found to be particularly important for learning robust representations.
  • Multi-task pre-training (MTP) demonstrated a significant improvement over single-task pre-training, and the full PTGB framework with MAML-styled training further enhanced accuracy and reduced variance.
  • The atlas mapping technique showed promising results in aligning virtual ROIs across different datasets, with observable overlaps.

Limitations and Future Work

  • Limitations of Atlas Alignment: While the atlas mapping showed promise, the paper notes that finding regions that overlap across all three investigated atlas templates was challenging, suggesting a limitation in the current unsupervised ROI alignment scheme.
  • Future Work: The authors suggest that future research could focus on modifying the current variance-based heuristic to improve cross-dataset alignment. They also express interest in further validating PTGB's generalizability as more brain network datasets become available. The effectiveness of different GNN architectures (GAT, GIN) was also explored, with GCN generally showing superior performance in this context, suggesting further investigation into architecture choices for brain network analysis.

Summary updated

Abstract

Human brains, controlling behaviors and cognition, are at the center of complex neurobiological systems. Recent studies in neuroscience and neuroimaging analysis have reached a consensus that interactions among brain regions of interest (ROIs) are driving factors for neural development and disorders. Graph neural networks as a powerful tool for analyzing graph-structured data are naturally applied to the analysis of brain networks. However, training of deep learning models including GNNs often requires a significant amount of labeled data. Due to the complicated data acquisition process and restrictions on data sharing, brain network datasets are still small compared to other domains (e.g., molecules, proteins). Moreover, real clinical tasks (e.g., mental disorder analysis) are often conducted on local datasets with even smaller scales and larger noises. To this end, we propose to leverage pre-training to capture the intrinsic brain network structures regardless of specific clinical outcomes. Specifically, we characterize the contributions in this work from two perspectives: (1) We design brain-network-oriented unsupervised pre-training techniques to utilize large-scale brain imaging studies without highly relevant task labels. (2) To facilitate effective knowledge transfer across studies with different ROI systems, we propose to develop a data-driven parcellation atlas mapping pipeline. The proposed pre-training techniques are validated with various GNN models. Extensive experiments demonstrate consistent improvement in performance as well as robustness.
BibTeX
@inproceedings{yang2023pre,
  author = {Yi Yang and Hejie Cui and Carl Yang},
  title = {{Pre-train Graph Neural Networks for Brain Network Analysis}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=I1Mdyc2Bg5x},
}
Figure