Learning Multimodal Graph-to-Graph Translation for Molecular Optimization

Wengong Jin, Kevin Yang, Regina Barzilay, Tommi S. JaakkolaarXiv 2018 · OpenReview

Abstract

We view molecular optimization as a graph-to-graph translation problem. The goal is to learn to map from one molecular graph to another with better properties based on an available corpus of paired molecules. Since molecules can be optimized in different ways, there are multiple viable translations for each input graph. A key challenge is therefore to model diverse translation outputs. Our primary contributions include a junction tree encoder-decoder for learning diverse graph translations along with a novel adversarial training method for aligning distributions of molecules. Diverse output distributions in our model are explicitly realized by low-dimensional latent vectors that modulate the translation process. We evaluate our model on multiple molecular optimization tasks and show that our model outperforms previous state-of-the-art baselines.

Introduction

In the field of drug discovery, finding a molecule that binds to a specific biological target is only the beginning. Often, a promising candidate comes with significant "baggage"—it might be poorly soluble, toxic, or difficult for the body to absorb. To transform these candidates into viable drugs, scientists must perform molecular optimization: tweaking the chemical structure to improve properties while preserving the core functional scaffold.

Traditionally, this has been a manual, rule-based process. However, researchers from MIT have proposed a more automated approach, framing molecular optimization as a graph-to-graph translation problem. By treating molecules as graphs and the optimization process as a form of "paraphrasing," they have developed a model capable of generating diverse, chemically valid, and optimized molecular structures.

The Challenge: Beyond Rule-Based Systems

Most traditional optimization methods rely on Matched Molecular Pair Analysis (MMPA). This technique uses a massive database of "if-then" rules (e.g., "swap this chemical group for that one to increase solubility"). While useful, MMPA is limited by the scope of its database and its inability to account for the broader context of the molecule.

On the other hand, deep generative models often attempt to build molecules atom-by-atom. While flexible, these models frequently produce chemically invalid structures or fail to maintain the structural essence of the starting molecule. Reinforcement learning (RL) approaches also exist but are often sample-inefficient and struggle to maintain structural similarity.

Proposed Method: Junction Tree Encoder-Decoder

The authors introduce the Variational Junction Tree Neural Network (VJTNN), which addresses these limitations through three key innovations.

Hierarchical Graph Translation

Instead of the error-prone atom-by-atom approach, the VJTNN uses a hierarchical assembly method. It first decomposes a molecule into a junction tree composed of pre-defined, chemically valid clusters such as rings and functional groups. The model learns to translate this tree structure first, and then fills in the specific atom-level connectivity. This approach ensures that the resulting molecules are always chemically valid.

The Junction Tree Encoder-Decoder pipeline: translating a molecular graph into a junction tree, modulating it with latent vectors, and decoding it back into a graph.

Multimodal Translation via Latent Spaces

A single molecule can often be optimized in multiple different ways. To capture this "multimodal" nature, the VJTNN incorporates low-dimensional latent vectors ($z$). By sampling different values for $z$, the model can "paraphrase" a single input molecule into a diverse set of optimized outputs. This is crucial for drug discovery, where researchers need multiple candidates to test in the laboratory.

Multimodal decoding: a single junction tree can be decoded into multiple distinct, chemically valid molecular graphs (1, 2, and 3).

Adversarial Scaffold Regularization

To ensure that the generated molecules align with the distribution of real-world drugs, the authors implemented an adversarial training strategy. A GAN-style discriminator is applied to the continuous hidden states of the junction tree decoder. This forces the model to generate scaffolds that are not only chemically valid but also "drug-like" in their overall architecture.

Experimental Evaluation

The model was tested on three primary tasks using the ZINC database: improving penalized logP (solubility), QED (drug-likeness), and DRD2 (biological activity).

The VJTNN significantly outperformed previous state-of-the-art models:

  • Success Rate: On the DRD2 activity task, VJTNN achieved a 78.4% success rate, vastly outperforming the RL-based GCPN (4.4%) and rule-based MMPA (46.4%).
  • Property Improvement: On the penalized logP task, it achieved property improvements of up to 3.55 units while maintaining high structural similarity to the original molecule.
  • Diversity: The model successfully generated a wide variety of valid "paraphrases" for single input molecules, providing a rich set of candidates for further testing.

Conclusion and Future Work

The VJTNN represents a significant step forward in automated molecular optimization. By combining junction tree representations with multimodal latent spaces and adversarial regularization, the model produces diverse, high-quality molecular candidates.

While highly effective, the model currently requires "parallel" data (pairs of similar molecules with different properties). Future research may focus on adapting this graph-to-graph translation framework for "unpaired" datasets, further expanding its utility in the complex landscape of drug discovery.

Reference: Jin, W., Yang, K., Barzilay, R., & Jaakkola, T. (2019). Learning Multimodal Graph-to-Graph Translation for Molecular Optimization. ICLR.

Summary updated

Abstract

We view molecular optimization as a graph-to-graph translation problem. The goal is to learn to map from one molecular graph to another with better properties based on an available corpus of paired molecules. Since molecules can be optimized in different ways, there are multiple viable translations for each input graph. A key challenge is therefore to model diverse translation outputs. Our primary contributions include a junction tree encoder-decoder for learning diverse graph translations along with a novel adversarial training method for aligning distributions of molecules. Diverse output distributions in our model are explicitly realized by low-dimensional latent vectors that modulate the translation process. We evaluate our model on multiple molecular optimization tasks and show that our model outperforms previous state-of-the-art baselines.
BibTeX
@misc{jin2018learning,
  author = {Wengong Jin and Kevin Yang and Regina Barzilay and Tommi S. Jaakkola},
  title = {{Learning Multimodal Graph-to-Graph Translation for Molecular Optimization}},
  year = {2018},
  eprint = {1812.01070},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1812.01070},
}
Figure