Graph neural networks to learn joint representations of disjoint molecular graphs

Chen Shao, Zhou Chen, Pascal FriedericharXiv 2022 · DOI · OpenReview

TL;DR

This paper introduces a new challenge for graph neural networks (GNNs): learning a single representation from multiple disjoint molecular graphs to predict a joint property, like a reaction's energy. They propose a new dataset of chemical reactions and show that while GNNs can learn these joint representations, they struggle with generalization to unseen molecules, highlighting a gap in current GNN capabilities for complex chemical tasks.

What Problem Does This Paper Address?

Traditionally, graph neural networks (GNNs) excel at learning representations for single, internally connected graphs. This is common in chemistry for tasks like predicting a molecule's solubility or toxicity. However, many real-world chemical problems involve multiple interacting molecules, where a single label (e.g., reaction energy, catalytic activity) depends on the combined properties of several disjoint molecular graphs. Existing GNNs aren't well-equipped to handle this "joint representation" problem for sets of independent graphs. This paper aims to bridge this gap by introducing a new task and dataset specifically designed to train and evaluate GNNs on such problems.

Methods

The authors adapt existing Message Passing Neural Network (MPNN) architectures for this new task. The core idea is to process each disjoint molecular graph individually and then combine their learned representations to predict a global label. They explore three input representations:

  • Disjoint Graph (DG): Each molecule is processed independently, and their final representations are concatenated.
  • Fully Connected Graph (FC): Molecules are treated as a single graph, with additional "non-physical" edges connecting nodes from different molecules.
  • Global Node (GN): A special "global node" is added, connected to all nodes in all input molecules. This aims to facilitate information flow between the disjoint graphs.

They also introduce a novel "reaction-based pooling layer" designed to better capture the joint properties of the input graphs.

Datasets

The paper introduces a new dataset constructed from chemical reactions, specifically focusing on the reaction between alcohols and acyl halides.

  • Dataset I: A combinatorial subset of reactions used for initial experiments.
  • Dataset II: A larger, full dataset containing 16,599 data points, each representing a pair of disjoint molecular graphs (one alcohol, one acyl halide). The label for each data point is the calculated reaction energy difference using the GFN-XTB software.

This dataset is designed to illustrate the challenge of learning joint representations from disjoint graphs, differing from existing datasets that typically focus on single molecular graphs.

Key Insights and Ideas

  • The "Joint Representation" Challenge: The core insight is that many important chemical tasks require understanding the collective behavior of multiple molecules, not just individual ones. This necessitates learning representations that capture inter-molecular interactions.
  • Global Node's Promise: The "Global Node" (GN) approach shows the most promise among the tested GNN architectures. This suggests that explicitly creating a mechanism for information to flow between disjoint graphs is crucial for learning these joint representations.
  • Generalization is Hard: The most significant finding is that while GNNs can learn to predict labels for combinations of molecules seen during training (interpolation), they struggle significantly to generalize to entirely new molecules not encountered in the training set. This highlights a critical limitation in current GNNs for real-world chemical applications.
  • MLP as a Strong Baseline: Surprisingly, a simple Multi-Layer Perceptron (MLP) trained on concatenated molecular fingerprints performs competitively, and sometimes better, than the GNNs in generalization tasks, suggesting that the current GNN architectures might not be optimally adapted for this specific problem.

Key Results

  • Experiment 1 (Interpolation): The Global Node (GN) GNN model achieved the best performance among the GNN variants, outperforming Disjoint Graph (DG) and Fully Connected Graph (FC) methods. The GN model showed a higher R² value (0.999) compared to the MLP (0.869).
  • Experiment 2 (Generalization): This experiment, where molecules in the test set were unseen during training, proved much harder. The MLP model outperformed the MPNN with a global node. Further modifications to the MPNN, like improved normalization and a modified global readout, were necessary to achieve performance slightly better than the MLP.

Limitations and Future Work

  • Generalization Gap: The primary limitation is the poor generalization performance of GNNs to unseen molecules. This suggests that current architectures may not effectively capture the underlying chemical principles that allow for transferability.
  • Hyperparameter Tuning: The authors acknowledge that the hyperparameters for the MPNNs might not have been optimally tuned for this specific task, which could contribute to the observed performance.
  • Future Work: The authors hope their work will stimulate further research into optimizing GNN architectures for disjoint graph representations. This includes exploring more sophisticated ways to model interactions between multiple molecules and developing better generalization strategies. They also suggest that this dataset is challenging and can serve as a benchmark for future advancements in this area.

Summary updated

Abstract

Graph neural networks are widely used to learn global representations of graphs, which are then used for regression or classification tasks. Typically, the graphs in such data sets are connected, i.e. each training sample consists of a single internally connected graph associated with a global label. However, there is a wide variety of yet unconsidered but application-relevant tasks, where labels are assigned to sets of disjoint graphs, which requires the generation of global representations of disjoint graphs. In this paper, we present a new data set with chemical reactions, which is illustrating this task. Each sample consists of a pair of disjoint molecular graphs and a joint label representing a scalar measure associated with the chemical reaction of the molecules. We show the initial results of graph neural networks that are able to solve the task within a combinatorial subset of the dataset but do not generalize well to the full data set and unseen (sub)graphs.
BibTeX
@misc{shao2022graph,
  author = {Chen Shao and Zhou Chen and Pascal Friederich},
  title = {{Graph neural networks to learn joint representations of disjoint molecular graphs}},
  year = {2022},
  eprint = {2210.09517},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.09517},
}
Figure