Molecular graph generation with Graph Neural Networks

Pietro Bongini, Monica Bianchini, Franco Scarselli

TL;DR

The paper introduces $MG^2N^2$ (Molecule Generative Graph Neural Network), a sequential framework that builds molecular graphs atom-by-atom using a modular set of Graph Neural Networks. By maximizing the structural information available at each step and allowing for independent module training, the model achieves state-of-the-art performance in generating unique and novel molecules on standard benchmarks like QM9 and ZINC.

What Problem Does This Paper Address?

Drug discovery is a notoriously slow and expensive process, often compared to finding a needle in a haystack the size of the universe. Computational methods, particularly deep learning, aim to navigate this "chemical space" by automatically generating candidate molecules.

While many early models treated molecules as text strings (SMILES notation), this often fails to capture the complex connectivity of a 3D molecule. Representing molecules as graphs (where atoms are nodes and bonds are edges) is more natural, but generating these graphs is difficult. Existing graph-based models often struggle with "mode collapse" (generating the same few patterns repeatedly) or fail to capture the full context of a molecule as it is being built. The authors sought a method that is both interpretable—allowing researchers to see the step-by-step construction—and flexible enough to learn complex chemical rules without hard-coding them.

Methods

The core of the paper is the $MG^2N^2$ architecture, which decomposes the complex task of graph generation into three simpler, sequential sub-tasks, each handled by a dedicated GNN module:

  1. M1 (Node Generation): Decides whether to add a new neighbor to the current "focus" node and, if so, what type of atom it should be (e.g., Carbon, Oxygen).
  2. M2 (First Edge Classification): Determines the bond type (single, double, or triple) between the focus node and the newly created node.
  3. M3 (Additional Node Linking): Scans the rest of the existing partial graph to see if the new node should form any additional bonds with other atoms. This step is performed in parallel to save time.
Figure 1: Flowchart of the generation algorithm. An example step is summarized, with the three GNN modules (M1, M2, M3), the three problems they are assigned to (P1, P2, P3), their inputs and their outputs. Grey nodes represent carbon atoms, while yellow nodes represent hydrogen atoms. Green edges stand for candidate edges, while black edges represent single bonds. C,H,N,O,F are the element symbols. Classes 1,2,3 represent single, double, and triple bonds, respectively. Red octagons stand for the stop decision (M1) or the do not generate this edge decision (M3)

The generation follows a Breadth-First Search (BFS) ordering, which ensures the graph stays connected and reduces the number of possible node permutations the model has to learn. To handle the discrete nature of choosing atoms and bonds while still allowing for gradient-based training, the authors employed Gumbel-softmax layers.

A key advantage of this modular approach is that each module can be trained independently. If you want to improve how the model handles bond types, you can retrain M2 without touching M1 or M3.

Datasets

The authors evaluated their model on two standard benchmarks:

  • QM9: A dataset of ~134k small organic molecules with up to 9 heavy atoms (C, O, N, F).
  • ZINC: A larger dataset of ~250k organic molecules with up to 38 heavy atoms, representing a more complex and diverse chemical space.

Key Insights and Ideas

  • Information Maximization: Unlike Recurrent Neural Networks (RNNs) that rely on a hidden state representing the history of decisions, $MG^2N^2$ uses a GNN to process the actual partial graph at every step. This ensures the model makes decisions based on the full structural context currently available.
  • Modularity as a Feature: The ability to swap or retrain modules independently makes the system highly maintainable and allows for targeted optimization of specific generative steps.
  • Learning the Rules vs. Hard-coding: While some models (like CGVAE) enforce chemical valence rules as hard constraints, $MG^2N^2$ successfully learns these rules from the data. This suggests the GNN architecture is powerful enough to internalize the "grammar" of chemistry.
  • Centrality-based Ordering: The authors found that ordering node expansion based on Freeman Betweenness Centrality (expanding more "central" atoms first) helped the model learn more effectively.

Key Results

The model was compared against several competitive baselines, including ChemVAE, MolGAN, and GraphVAE.

  • QM9 Performance: $MG^2N^2$ achieved a Novelty score of 1.000 (meaning every valid molecule generated was new and not in the training set) and outperformed all baselines on the VUN score (a composite metric of Validity, Uniqueness, and Novelty).
  • ZINC Performance: On the more challenging ZINC dataset, the model maintained high novelty (1.000) and significantly outperformed standard VAE-based graph generators in validity.
  • Chemical Fidelity: Qualitative analysis showed that the generated molecules closely matched the distributions of chemical properties (like molecular weight and logP) found in the original datasets, indicating that the model isn't just generating random graphs, but realistic molecules.

Limitations and Future Work

The authors noted a few areas for improvement:

  • Independent Training Assumption: Currently, modules are trained assuming the other modules always provide the "ground truth." This might limit the model's ability to recover from early mistakes during the generation phase.
  • Edge Dependencies: In module M3, additional edges are predicted in parallel, which ignores potential dependencies between those specific bonds.
  • Scaling to Large Molecules: While it performs well on ZINC, the validity of generated molecules tends to decrease as the number of atoms increases. The authors suggest that a fragment-based approach (adding groups of atoms instead of single atoms) could be a promising future direction to handle even larger structures.

Summary updated

Abstract

Drug Discovery is a fundamental and ever-evolving field of research. The design of new candidate molecules requires large amounts of time and money, and computational methods are being increasingly employed to cut these costs. Machine learning methods are ideal for the design of large amounts of potential new candidate molecules, which are naturally represented as graphs. Graph generation is being revolutionized by deep learning methods, and molecular generation is one of its most promising applications. In this paper, we introduce a sequential molecular graph generator based on a set of graph neural network modules, which we call MG^2N^2. At each step, a node or a group of nodes is added to the graph, along with its connections. The modular architecture simplifies the training procedure, also allowing an independent retraining of a single module. Sequentiality and modularity make the generation process interpretable. The use of graph neural networks maximizes the information in input at each generative step, which consists of the subgraph produced during the previous steps. Experiments of unconditional generation on the QM9 and Zinc datasets show that our model is capable of generalizing molecular patterns seen during the training phase, without overfitting. The results indicate that our method is competitive, and outperforms challenging baselines for unconditional generation.
BibTeX
@article{bongini2021molecular,
  author = {Pietro Bongini and Monica Bianchini and Franco Scarselli},
  title = {{Molecular graph generation with Graph Neural Networks}},
  journal = {Neurocomputing},
  year = {2021},
  doi = {10.1016/j.neucom.2021.04.039},
}
Figure