Molecular Attributes Transfer from Non-Parallel Data

Shuangjia Zheng, Ying Song, Zhang Pan, Chengtao Li, Le Song, Yuedong YangarXiv 2021 · OpenReview

TL;DR

This paper tackles the challenge of optimizing chemical molecules for desired properties without requiring paired examples of original and optimized molecules. They achieve this by framing molecular optimization as a "style transfer" problem, developing a novel generative model that learns to modify molecular attributes (like toxicity or synthesizability) while preserving essential structural content, outperforming existing methods.

What Problem Does This Paper Address?

Drug development hinges on optimizing chemical molecules for specific properties (e.g., efficacy, safety, manufacturability). Traditional methods often struggle with the vastness of chemical space and the lack of paired data (an original molecule and its perfectly optimized counterpart). Existing deep learning approaches either require predefined attribute functions, are computationally expensive, or fail to preserve crucial molecular characteristics. This paper addresses the critical need for a method that can effectively modify molecular properties using only non-parallel datasets (collections of molecules with different attributes but no direct correspondence), a more realistic scenario in real-world drug discovery.

Methods

The core of this work is a novel generative model that treats molecular optimization as a "style transfer" problem. Key techniques include:

  • Style Transfer Formulation: The problem is reframed as transferring a "style" (e.g., toxicity, synthesizability) from one set of molecules to another, while preserving the "content" (e.g., molecular weight, ring count).
  • Auxiliary Guided-VAE: A Variational Autoencoder (VAE) is pre-trained to learn a chemically interpretable latent representation. This VAE is "guided" by auxiliary predictors that learn to associate latent variables with specific molecular attributes, ensuring content preservation.
  • Generative Flow for Style Latent Space: To capture richer stylistic features, a generative flow technique is employed. This allows the model to learn a more expressive latent space for molecular styles by considering a set of molecules with the same style.
  • Adversarial Training: An adversarial discriminator is used to differentiate between styles, forcing the generator to produce molecules with the target style. This is performed in the continuous latent space, avoiding issues with discrete molecular representations.
  • Auxiliary Losses:
    • Self Reconstruction Loss: Ensures the latent representation of input molecules can be reconstructed from their source style.
  • Cycle Consistency Loss: Encourages generated molecules to preserve the content of the original input molecule by ensuring that a generated molecule's latent representation can be reconstructed back to the original content latent space.

Datasets

The experiments were conducted on two molecular attribute transfer tasks:

  • Toxicity Modification: Molecules were modified to change their mutagenic toxicity. The toxicity prediction model was trained on a dataset derived from ZINC lead-like compounds, with approximately 400K toxic and 400K non-toxic molecules, split into train/dev/test sets.
  • Synthesizability Modification: Molecules were modified to decrease their synthetic complexity. This involved transforming molecules with a synthetic accessibility (SA) score in the range [5,8] (hard to synthesize) to the range [0,2.5] (trivial to synthesize). A similar split of 400K trivial and 400K hard molecules from ZINC lead-like dataset was used.

Key Insights and Ideas

  • Molecular Optimization as Style Transfer: This is a novel framing that allows leveraging techniques from text style transfer for molecular design, opening up new avenues for research.
  • Handling Non-Parallel Data: The ability to learn from non-parallel data is a significant practical advantage, as collecting paired molecular data is often infeasible.
  • Disentangling Content and Style: The model's success in preserving molecular content while changing attributes highlights the effectiveness of its approach to disentangling these two aspects in the latent space.
  • Adversarial Regularization in Continuous Latent Space: This strategy effectively addresses the discrete nature of molecular representations (like SMILES strings), which is a common challenge in generative molecular modeling.

Key Results

  • Superior Performance: The proposed model, MolStyle, significantly outperforms several state-of-the-art methods on both toxicity modification and synthesizability improvement tasks, as measured by metrics like Property Improvement (Imp.), Physicochemical Similarity Score (PSS), Validity (Val.), Success Rate (SR), and Geometric Mean (GM).
  • High Content Preservation: MolStyle achieves a high PSS score, indicating that generated molecules retain key physicochemical properties of the original input.
  • Effective Style Control: The model successfully modifies the target attributes (toxicity and synthesizability) as demonstrated by property improvement metrics.
  • Structural Alert Reduction: In toxicity modification, MolStyle showed a significant reduction in the presence of toxic substructures compared to baselines, suggesting it removes unfavorable chemical motifs.
  • High Validity Rate: The model generates chemically valid molecules with a high success rate (98.6%).

Limitations and Future Work

  • Potential for Exploiting Predictor Loopholes: The authors acknowledge that performance is often evaluated using learned property predictors, which could be exploited by the model. To address this, they introduced a structural alert-based metric for toxicity evaluation.
  • Activity Cliffs: The paper mentions that a small modification in molecular structure can lead to large changes in chemical properties (activity cliffs), which can make attribute transfer challenging. While their model performs well, this remains an inherent difficulty in the field.
  • Future Work: The paper suggests that the framework could be extended to handle more complex molecular optimization tasks and potentially incorporate more sophisticated generative flow techniques for even richer style representations. The success in disentangling content and style also opens doors for exploring multi-objective optimization with greater control.

Summary updated

Abstract

Optimizing chemical molecules for desired properties lies at the core of drug development. Despite initial successes made by deep generative models and reinforcement learning methods, these methods were mostly limited by the requirement of predefined attribute functions or parallel data with manually pre-compiled pairs of original and optimized molecules. In this paper, for the first time, we formulate molecular optimization as a style transfer problem and present a novel generative model that could automatically learn internal differences between two groups of non-parallel data through adversarial training strategies. Our model further enables both preservation of molecular contents and optimization of molecular properties through combining auxiliary guided-variational autoencoders and generative flow techniques. Experiments on two molecular optimization tasks, toxicity modification and synthesizability improvement, demonstrate that our model significantly outperforms several state-of-the-art methods.
BibTeX
@misc{zheng2021molecular,
  author = {Shuangjia Zheng and Ying Song and Zhang Pan and Chengtao Li and Le Song and Yuedong Yang},
  title = {{Molecular Attributes Transfer from Non-Parallel Data}},
  year = {2021},
  eprint = {2111.15146},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.15146},
}
Figure