Diverse Hits in De Novo Molecule Design: Diversity-Based Comparison of Goal-Directed Generators

Philipp Renz, Sohvi Luukkonen, Günter KlambauerJournal of Chemical Information and Modeling 2024 · DOI

TL;DR

This paper tackles the challenge of generating diverse sets of high-scoring drug-like molecules using AI. They introduce a more robust benchmark that considers computational constraints and a better diversity metric (#Circles), finding that SMILES-based autoregressive models outperform graph-based and genetic algorithms in this crucial task.

What Problem Does This Paper Address?

Discovering new drug candidates is a complex process, and "de novo" molecular design aims to use AI to generate novel molecules with desired properties (like efficacy and low toxicity). However, many current AI generators suffer from "mode collapse," producing very similar molecules, or they over-optimize to imperfect scoring functions, leading to a lack of true diversity. This limits their usefulness in exploring the vast chemical space and finding truly novel compounds. Furthermore, previous benchmarks often overlooked the practical constraints of computational cost and used diversity metrics that didn't accurately capture chemical space coverage.

Methods

The authors propose a new benchmark setup for evaluating goal-directed molecule generators. Key aspects include:

  • #Circles Diversity Metric: Instead of simpler metrics like uniqueness or internal diversity (which can be misleading), they use #Circles. This metric counts the number of generated molecules that are sufficiently distinct from each other, providing a better measure of chemical space coverage.
  • Goal-Directed Optimization Framework: Generators are evaluated within a framework that mimics real-world drug discovery, where computational resources are limited.
  • Computational Constraints: The benchmark includes two specific constraints:
  • Sample Limit: A maximum of 10,000 scoring function evaluations.
  • Time Limit: A maximum of 600 seconds of computation time.
  • Scoring Functions: They use Random Forest classifiers trained on three well-established bioactivity datasets (JNK3, GSK3β, and DRD2) as scoring functions. These are augmented with property filters (molecular weight, logP, and idiosyncratic substructures) and a diversity filter (to prevent generating molecules too similar to already found ones).

Datasets

The evaluation was performed on three bioactivity datasets:

  • JNK3
  • GSK3β
  • DRD2

These datasets were used to train Random Forest classifiers that served as the scoring functions for the generative models. The authors also mention using the GuacaMol dataset for virtual screening baselines.

Key Insights and Ideas

  • Diversity Metrics Matter: The #Circles metric is crucial for accurately assessing the diversity of generated molecule sets, as traditional metrics can be easily gamed or fail to capture true chemical space coverage.
  • Computational Constraints are Realistic: Evaluating generators under realistic computational budgets (time and sample limits) is essential, as performance can drastically change compared to unconstrained scenarios.
  • SMILES-Based Models Shine: Autoregressive models operating on SMILES strings generally outperform graph-based models and genetic algorithms in generating diverse, high-scoring molecules.
  • Single-Molecule vs. Diverse Optimization: A model's strong performance in single-molecule optimization tasks does not guarantee good performance in diverse generation tasks.
  • Trade-off Between Diversity and Drug-Likeness: Some models achieve high diversity at the expense of drug-likeness (e.g., generating larger, less drug-like molecules).

Key Results

  • Performance Variability: The number of diverse hits found varied significantly across different models, tasks, and computational constraints, ranging from a few molecules to thousands.
  • SMILES Models Lead: SMILES-based autoregressive models, particularly AugMemory and LSTM-HC, consistently ranked high in generating diverse hits, especially under time constraints.
  • Task and Constraint Dependence: Performance was highly dependent on the specific target protein (GSK3β generally yielded more diverse hits than DRD2/JNK3) and the computational constraint applied.
  • Graph-Based and Genetic Algorithms Lag: Graph-based models and genetic algorithms generally occupied lower ranks, with GraphGA being an exception that outperformed virtual screening baselines.
  • LSTM-HC Versatility: LSTM-HC was identified as a versatile algorithm, achieving on average 84% of the top performance across all tested settings.

Limitations and Future Work

  • Scoring Function Limitations: The study acknowledges that the scoring functions used (Random Forest classifiers) are imperfect and that more accurate, computationally expensive methods like docking could be incorporated in future work.
  • Synthesizability and Compute Budgets: The benchmark setup does not cover all aspects of practical drug discovery, such as synthesizability or varying compute budgets, which can differ significantly in real-world applications.
  • Generalizability: While the authors believe their findings will generalize, they note that the specific application and available resources are critical factors in choosing a generative model.
  • Future Directions: The authors suggest that their benchmark setup will be useful for future studies in the field, implying continued research into improving generative models for diverse de novo design.

Summary updated

BibTeX
@article{renz2024diverse,
  author = {Philipp Renz and Sohvi Luukkonen and Günter Klambauer},
  title = {{Diverse Hits in De Novo Molecule Design: Diversity-Based Comparison of Goal-Directed Generators}},
  journal = {Journal of Chemical Information and Modeling},
  year = {2024},
  doi = {10.1021/acs.jcim.4c00519},
}
Figure