Pre-Training Graph Neural Networks on Molecules by Using Subgraph-Conditioned Graph Information Bottleneck

Van Thuy Hoang, O-Joun LeeAAAI 2025 · DOI

TL;DR

This paper introduces S-CGIB, a novel self-supervised pre-training method for Graph Neural Networks (GNNs) on molecules. S-CGIB effectively learns both a compressed "graph core" representation and identifies "significant subgraphs" (functional groups) without requiring human annotations, leading to improved performance on downstream molecular property prediction tasks.

What Problem Does This Paper Address?

Predicting molecular properties using Graph Neural Networks (GNNs) has shown great promise in computational chemistry. However, the effectiveness of GNNs heavily relies on the availability of large, labeled molecular datasets, which are often scarce and expensive to obtain. While pre-training strategies have emerged to mitigate this data scarcity, existing methods face limitations. Node-level pre-training often fails to capture the crucial high-order structures like functional groups. Contrastive learning methods can sometimes disrupt molecular structures through augmentations. Subgraph-level strategies, while better at capturing functional groups, typically rely on human annotations or prior knowledge to identify these substructures, limiting their generalizability. The core challenge is to develop a pre-training method that can learn both robust, well-distinguished graph-level representations and automatically discover important molecular substructures (functional groups) without any external supervision.

Methods

The paper proposes Subgraph-conditioned Graph Information Bottleneck (S-CGIB), a self-supervised pre-training framework. The core idea is to compress an input molecule's graph representation into a "graph core" that retains essential information for reconstructing the original graph, but only when conditioned on identified "significant subgraphs."

Here's a breakdown of the key components:

  • Graph Information Bottleneck (GIB) Principle: The foundation is the GIB principle, which aims to compress an input graph into a core subgraph that preserves sufficient information for a downstream task (like predicting labels) while minimizing redundancy.
  • Subgraph-Conditioned Graph Information Bottleneck (S-CGIB): To adapt GIB for self-supervised learning without labels, S-CGIB modifies the objective. Instead of predicting labels, it focuses on reconstructing the original graph. The "conditioning" on significant subgraphs is crucial.
  • Graph Core Extraction: The model first uses a GNN encoder to generate initial node embeddings. Then, it injects noise into these embeddings to create a compressed representation, the "graph core" (Z). This process is designed to distill important nodes and discard irrelevant ones.
  • Subgraph Learning: To identify "significant subgraphs" (potential functional groups) without prior knowledge, the method generates candidates by extracting "ego networks" (k-hop neighborhoods around each node). An attention-based mechanism then learns to weigh the importance of these candidates in relation to the graph core, effectively highlighting significant subgraphs.
  • Joint Optimization: The model is trained by jointly optimizing two objectives:
  • Conditional Reconstruction Loss (Lrec): Encourages the graph core to reconstruct the original graph structure, conditioned on the identified significant subgraphs.
  • Compression Loss (LMI): Minimizes the mutual information between the graph core and the original graph, ensuring compression.
  • Contrastive Learning Loss (Lcon): Further enhances the distinctiveness of the graph core by ensuring high mutual information with its own graph compared to other graphs.

Datasets

The paper uses a variety of datasets for pre-training and fine-tuning:

  • Pre-training Datasets: 300k unlabeled molecules from PCQ4Mv2, QM9, and mol-PCBA.
  • Fine-tuning Datasets:
    • Biophysics: mol-HIV, mol-PCBA, BACE
    • Physiology: BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV
    • Physical Chemistry: ESOL, FreeSolv, Lipophilicity
    • Bioinformatics: Mutagenicity, NCI1, NCI109
    • Quantum Mechanics: PCQ4Mv2, QM9
  • Large Molecular Graphs: Peptide-func and Peptide-struct.
  • Interpretability Datasets: Mutagenicity, BENZENE, Alkane Carbonyl, and Fluoride Carbonyl.

Key Insights and Ideas

  • Unsupervised Functional Group Discovery: The most significant insight is the ability to automatically discover and leverage "significant subgraphs" (functional groups) without any human-provided labels or predefined dictionaries. This is achieved through the attention mechanism between the graph core and ego network candidates.
  • Graph Core as a Bottleneck for Reconstruction: The "graph core" isn't just a compressed representation; it's specifically designed to be sufficient for reconstructing the original graph when conditioned on the identified functional groups. This creates a powerful self-supervision signal.
  • Bridging Graph Core and Subgraphs: The paper elegantly combines the idea of a compressed graph core with the identification of important substructures. This dual focus allows for both robust global representations and detailed local structural understanding.
  • Interpretability Through Learned Subgraphs: The method's ability to identify meaningful subgraphs also contributes to model interpretability, allowing researchers to understand why a model makes certain predictions by highlighting the important functional groups.

Key Results

  • Superior Performance: S-CGIB consistently outperformed existing node-level, contrastive, and subgraph-level pre-training methods across 10 out of 11 graph classification datasets, achieving the best performance.
  • Regression Tasks: It also showed superior performance on three regression benchmarks (Table 3).
  • Large Molecules: S-CGIB demonstrated strong performance on large molecular graphs (Peptides-func and Peptides-struct), outperforming baselines by a significant margin (e.g., 12.3% improvement over GraphFP on Peptides-func).
  • Efficiency: The pre-trained model converged faster and required less training/validation time on downstream tasks.
  • Interpretability: Qualitative visualizations (Figure 3) and quantitative metrics (Table 4, Table 6) showed that S-CGIB's learned explanations (identified subgraphs) were more accurate and reliable compared to baselines.
  • Ablation Studies: Both the graph core and subgraph learning components were shown to be crucial for S-CGIB's performance, with subgraph learning being slightly more impactful for molecular property prediction.

Limitations and Future Work

  • Subgraph Size: The paper notes that subgraph size is important; very small subgraphs might not capture enough information, while very large ones could introduce noise. The optimal size (k ≤ 3) was found to be effective.
  • Domain Adaptation: While the pre-trained model generalizes well, the authors suggest that further unsupervised domain adaptation can improve performance on downstream datasets with distinct node attribute characteristics. They propose a node feature reconstruction loss for this purpose.
  • Future Work: The authors suggest exploring different graph encoders, subgraph sizes, and GIN layer configurations, which they briefly investigate in the sensitivity analysis. The core framework of S-CGIB, however, is presented as a robust foundation for future research in molecular GNN pre-training.

Summary updated

BibTeX
@inproceedings{hoang2025pre,
  author = {Van Thuy Hoang and O-Joun Lee},
  title = {{Pre-Training Graph Neural Networks on Molecules by Using Subgraph-Conditioned Graph Information Bottleneck}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i16.33891},
}
Figure