Graph Energy-based Model for Substructure Preserving Molecular Design

Ryuichiro Hataya, Hideki Nakayama, Kazuki YoshizoearXiv 2021 · OpenReview

Abstract

It is common practice for chemists to search chemical databases based on substructures of compounds for finding molecules with desired properties. The purpose of de novo molecular generation is to generate instead of search. Existing machine learning based molecular design methods have no or limited ability in generating novel molecules that preserves a target substructure. Our Graph Energy-based Model, or GEM, can fix substructures and generate the rest. The experimental results show that the GEMs trained from chemistry datasets successfully generate novel molecules while preserving the target substructures. This method would provide a new way of incorporating the domain knowledge of chemists in molecular design.

Introduction

In drug discovery and materials science, researchers rarely design molecules from a blank slate. Instead, they typically iterate on "lead compounds"—molecules with known beneficial properties—by modifying specific functional groups while keeping the core scaffold intact. This process, known as substructure-preserving design, is fundamental to medicinal chemistry.

However, modern generative models like Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) struggle with these types of "hard constraints." Most are designed to generate entire molecular graphs in a single pass, making it difficult to enforce the preservation of a specific user-defined substructure. To bridge this gap, we introduce the Graph Energy-based Model (GEM), a framework that treats molecular design as an energy minimization problem, allowing for flexible, constrained generation.

Methodology: The Graph Energy-based Model (GEM)

Unlike traditional generative models that learn a mapping from a latent space to a data space, an Energy-Based Model (EBM) learns a scalar energy function $E_\theta(G)$ that assigns low energy to realistic molecular graphs and high energy to unrealistic ones.

Training and Generation

The GEM framework operates in two distinct phases: training and generation. During training, the model learns to shape the energy landscape using a dataset of known molecules. During generation, the model starts with a random initial state (noise) and uses Stochastic Gradient Langevin Dynamics (SGLD) to iteratively "walk" toward regions of lower energy, effectively hallucinating a valid molecular structure.

The GEM training and generation loop. During training, the Graph NN learns the energy function. During generation, the model minimizes energy to produce a molecular graph.

Graph Representation and Symmetrization

Molecules are represented using two primary tensors: a Feature Tensor (X), which encodes atom types (e.g., Carbon, Nitrogen, Oxygen), and an Adjacency Tensor (A), which encodes bond types and connectivity. Because molecular graphs are discrete, GEM employs dequantization by adding small amounts of continuous noise to these tensors during training. Furthermore, to maintain chemical consistency, the model uses a modified Relational Graph Convolutional Network (RGCN) with gradient symmetrization, ensuring that the relationship between two atoms is identical regardless of the direction of the edge in the adjacency tensor.

Substructure Preservation via Masking

The defining feature of GEM is its ability to preserve substructures. Because generation is an iterative refinement process, we can apply a mask to the parts of the tensors representing the target substructure. By freezing these values at every step of the SGLD sampling process, the model is forced to optimize the rest of the molecule around the fixed core.

Overview of the substructure preservation workflow. A target substructure is fixed in the tensors, and GEM generates the remaining molecular components.

Experimental Results

The model was evaluated on the QM9 (small organic molecules) and ZINC-250k (drug-like molecules) datasets.

Preservation and Novelty

GEM demonstrated a high success rate in preserving complex substructures like benzene rings and piperazine while generating chemically diverse completions. On the QM9 dataset, the model achieved a novelty score of 92.4%, indicating that the generated molecules were not simply memorized from the training set but were genuine de novo designs.

Property Optimization

Beyond simple generation, GEM can optimize for specific properties by incorporating a regression loss into the energy function. For example, when tasked with improving the Quantitative Estimate of Drug-likeness (QED), GEM successfully transformed a simple benzene ring (QED 0.44) into more complex, drug-like derivatives with scores exceeding 0.60. It also showed significant improvements in penalized logP (solubility and synthetic accessibility) compared to random completion baselines.

Limitations and Future Outlook

While GEM excels at constrained generation, it faces challenges with validity when generating molecules entirely from scratch. On the ZINC dataset, the validity of unconstrained outputs was lower than state-of-the-art GANs, likely due to the vast search space of the adjacency tensors.

Future work will focus on implementing GEM within a latent space to improve efficiency and incorporating explicit chemical valency rules to ensure that all generated bonds are chemically feasible. Despite these challenges, GEM represents a significant step toward integrating human domain knowledge directly into the AI-driven molecular design process.

Summary updated

Abstract

It is common practice for chemists to search chemical databases based on substructures of compounds for finding molecules with desired properties. The purpose of de novo molecular generation is to generate instead of search. Existing machine learning based molecular design methods have no or limited ability in generating novel molecules that preserves a target substructure. Our Graph Energy-based Model, or GEM, can fix substructures and generate the rest. The experimental results show that the GEMs trained from chemistry datasets successfully generate novel molecules while preserving the target substructures. This method would provide a new way of incorporating the domain knowledge of chemists in molecular design.
BibTeX
@misc{hataya2021graph,
  author = {Ryuichiro Hataya and Hideki Nakayama and Kazuki Yoshizoe},
  title = {{Graph Energy-based Model for Substructure Preserving Molecular Design}},
  year = {2021},
  eprint = {2102.04600},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.04600},
}
Figure