Transformer-based Objective-reinforced Generative Adversarial Network to Generate Desired Molecules

Chen Li, Chikashige Yamanaka, Kazuma Kaitoh, Yoshihiro YamanishiIJCAI 2022 · DOI

TL;DR

This paper introduces TransORGAN, a novel generative model that uses a Transformer architecture within an objective-reinforced Generative Adversarial Network (GAN) framework to create new molecules. By combining the Transformer's ability to capture rich semantic features with reinforcement learning and a novel "variant SMILES" approach, TransORGAN generates molecules that are not only chemically valid but also unique, novel, and diverse, addressing key challenges in drug discovery.

What Problem Does This Paper Address?

The process of discovering new drugs is notoriously long, expensive, and complex. While Artificial Intelligence (AI) and deep learning are increasingly being used to accelerate this process, generating novel and useful molecular structures remains a significant challenge. Existing deep generative models, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), often struggle to fully capture the intricate semantic features of molecules represented as sequences (like SMILES strings). Furthermore, GANs are prone to "mode collapse," leading to a lack of diversity in generated molecules. Traditional GANs also face difficulties in generating discrete data like SMILES strings, and their training can be unstable. This paper aims to overcome these limitations by proposing a more robust and effective generative model for de novo molecular design.

Methods

The core of TransORGAN is a Transformer-based generator. Unlike traditional RNNs, Transformers utilize a self-attention mechanism that allows for parallel processing and a deeper understanding of long-range dependencies within the SMILES sequence, capturing richer semantic information. To address the discrete nature of SMILES generation and improve training stability, TransORGAN employs an objective-reinforced learning (RL) approach, similar to SeqGAN and ORGAN.

A key innovation is the use of "variant SMILES." Since a single molecule can have multiple valid SMILES representations, the model generates these variants during training. This helps the Transformer learn syntactic and semantic rules more effectively and increases the diversity of generated molecules, preventing the model from overfitting to a single representation.

The discriminator in TransORGAN is trained to distinguish between real and generated molecules and also provides rewards that guide the generator's policy update. An objective-reinforced penalty is incorporated to further encourage the generation of diverse molecules. The training process involves alternating between training the generator and the discriminator, with a teacher-forcing mechanism used to improve policy gradient stability during training.

Datasets

The experiments were conducted using a subset of the ZINC chemical dataset. This dataset contains molecules represented by SMILES strings. The specific subset used comprised 5,000 randomly selected molecules with up to nine heavy atoms, drawn from the GDB-17 universe.

Key Insights and Ideas

  • Transformers for Molecular Semantics: The paper highlights that Transformer architectures, with their self-attention mechanisms, are significantly better at capturing the rich semantic nuances of molecular sequences (SMILES) compared to traditional RNNs. This is crucial for generating molecules with desired properties.
  • Variant SMILES for Diversity: The introduction of "variant SMILES" is a clever way to augment the training data and force the model to learn more robust representations, directly combating mode collapse and boosting molecular diversity.
  • Objective Reinforcement for Targeted Generation: By integrating objective-reinforced learning, the model can be guided not just to generate valid molecules but also to optimize for specific desirable chemical properties (like solubility or drug-likeness), making the generated molecules more practically useful.
  • Balancing Validity and Novelty: The results suggest that TransORGAN strikes a better balance between generating chemically valid molecules and ensuring those molecules are novel and unique, a critical aspect for drug discovery where novelty is paramount.

Key Results

TransORGAN demonstrated strong performance across several evaluation metrics:

  • Uniqueness, Novelty, and Diversity: TransORGAN significantly outperformed baseline models like Naïve RL and ORGAN in terms of uniqueness, novelty, and diversity. For example, when optimizing for solubility, TransORGAN achieved a diversity score of 0.650, substantially higher than Naïve RL (0.339) and ORGAN (0.358).
  • Drug-Likeness: The model showed improved drug-likeness, with a higher mean QED score (0.62) compared to Naïve RL (0.55) and ORGAN (0.50).
  • Comparison to Baselines: While some baseline models like Random Sampler and JT-VAE achieved 100% validity, they suffered from very low novelty and uniqueness. TransORGAN achieved competitive validity (around 74-75% for objective-reinforced tasks) while excelling in uniqueness, novelty, and diversity.
  • Qualitative Improvement: The paper notes that molecules generated by TransORGAN often contained substructures commonly found in approved drugs, whereas ORGAN and Naïve RL sometimes produced unnatural or unstable structures.

Limitations and Future Work

The authors acknowledge a key limitation: optimizing the hyperparameters for the trade-off between likelihood and RL (λ) and the Monte Carlo search times (N) can be challenging. They suggest that these parameters significantly influence the model's performance and require careful tuning.

Future work could involve:

  • Investigating the impact of different values for λ and N on molecular generation performance.
  • Exploring more sophisticated methods for hyperparameter optimization.
  • Potentially extending the model to handle other molecular representations or tasks beyond SMILES generation.

Summary updated

BibTeX
@inproceedings{li2022transformer,
  author = {Chen Li and Chikashige Yamanaka and Kazuma Kaitoh and Yoshihiro Yamanishi},
  title = {{Transformer-based Objective-reinforced Generative Adversarial Network to Generate Desired Molecules}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/539},
}
Figure