Generating Novel Leads for Drug Discovery Using LLMs with Logical Feedback

Shreyas Bhat Brahmavar, Ashwin Srinivasan, Tirtharaj Dash, Sowmya Ramaswamy Krishnan, Lovekesh Vig, Arijit Roy, Raviprasad AduriAAAI 2024 · DOI

TL;DR

This paper introduces LMLF, a novel method that guides Large Language Models (LLMs) in drug discovery by separating logical constraints from natural language queries. This approach allows for more effective generation of potential drug leads with higher binding affinities and improved novelty compared to standard LLM prompting.

What Problem Does This Paper Address?

The paper tackles the challenge of effectively leveraging the vast knowledge embedded within Large Language Models (LLMs) for complex scientific tasks like drug discovery. While LLMs excel at natural language interaction, extracting specific, reliable information for scientific applications often requires intricate "prompt engineering." This process is subjective, difficult to reproduce, and demands specialized skills. The authors highlight a "reverse bottleneck" where human experts struggle to access and control the machine's knowledge. Specifically, in early-stage drug design, computational chemists need to generate novel molecules that bind effectively to specific biological targets. This requires translating expert knowledge, often expressed in logical forms (e.g., molecular weight constraints, binding affinity thresholds), into prompts that LLMs can understand and act upon. The current reliance on natural language prompts makes this process inefficient and inconsistent.

Methods

The core contribution is the Language Models with Logical Feedback (LMLF) procedure. LMLF reframes the LLM prompt by splitting it into two components:

  • Domain-specific constraints (C): These are expressed in a formal logical language (e.g., "molecular weight between 200 and 700," "binding affinity is -8 or less").
  • Domain-independent query (Q): A simple natural language request (e.g., "Generate valid SMILES string for n molecules that are labelled '1' and are not found in any known database").

LMLF operates iteratively:

  1. Prompt Assembly: The LLM receives a prompt combining background knowledge (B), logical constraints (C), and the query (Q).
  2. Instance Generation: The LLM generates candidate molecules.
  3. Logical Feedback: These generated molecules are evaluated against the logical constraints. Molecules that satisfy the constraints are "labelled" positively.
  4. Constraint Generalization: Based on the feedback, the logical constraints are automatically refined (generalized) for the next iteration. This process is inspired by the MIMIC algorithm and leverages "constraint-based labelling" and "constraint-generalisation" properties.

This iterative refinement of logical constraints, rather than manual prompt tweaking, allows the LLM to progressively focus its generation towards molecules that meet the desired criteria.

Datasets

The paper evaluates LMLF using two well-known biological targets:

  • JAK2 (Janus Kinase 2): A dataset of 4100 molecules with labels (3700 active).
  • DRD2 (Dopamine Receptor D2): A dataset of 4070 molecules with labels (3670 active).

These datasets were sourced from ChEMBL and filtered based on IC50 values and docking scores. The paper does not introduce new benchmarks but uses these established datasets for evaluation.

Key Insights and Ideas

  • Decoupling Logic from Language: The most significant insight is the power of separating formal logical constraints from natural language queries. This allows for a more systematic and reproducible way to guide LLMs, moving beyond the art of prompt engineering.
  • Iterative Refinement is Key: The iterative nature of LMLF, where logical feedback drives the refinement of constraints, is crucial for progressively improving the quality and relevance of generated molecules.
  • LLMs as Knowledge Repositories: The work underscores that LLMs, when properly guided, can act as sophisticated knowledge repositories for complex scientific domains like drug discovery.
  • Neuro-Symbolic Approach: LMLF represents a promising "neuro-symbolic" approach, combining the pattern-matching capabilities of neural networks (LLMs) with the precision of symbolic logic.

Key Results

The experiments demonstrate that LMLF significantly improves the generation of potential drug leads:

  • Higher Binding Affinities: Molecules generated using LMLF consistently showed higher binding affinities compared to those generated by LLMs without logical feedback. For JAK2, the mean binding affinity increased from 6.50 (no constraints) to 7.74 (with LMLF and full constraints) for GPT-3. For DRD2, it increased from 6.48 to 7.66.
  • Improved Performance with Constraints: Adding target-agnostic constraints improved performance, and further adding target-specific constraints yielded even better results, quantitatively supporting the conjecture that logical feedback is beneficial.
  • Novelty and Efficacy: A qualitative assessment by a computational chemist indicated that LMLF-generated compounds were potentially novel inhibitors. For JAK2, 15% of generated molecules contained JAK2-selective functional groups. For DRD2, 40% contained DRD2-selective functional groups.
  • Comparison of LLMs: While both GPT-3 and PaLM benefited from LMLF, GPT-3 showed slightly better performance in terms of identified effective and novel inhibitors, though the differences were not statistically significant.
  • Comparison to Baselines: LMLF-generated molecules outperformed existing baseline methods like VAE-GNN in terms of binding affinities.

Limitations and Future Work

  • Scope of Generalization: The current generalization of constraints is restricted to numeric inequalities. Future work could explore more complex logical structures.
  • LLM Capabilities: The effectiveness of LMLF is still dependent on the underlying capabilities of the LLM.
  • Computational Cost: While LMLF aims for efficiency, the iterative nature and external evaluation steps can still be computationally intensive.
  • Future Directions: The authors suggest that LMLF could be adapted for other generative models beyond LLMs and applied to a wider range of scientific discovery tasks. They envision LMLF as a foundation for "closed-loop scientific discovery" where AI systems can autonomously refine hypotheses and conduct experiments.

Summary updated

BibTeX
@inproceedings{brahmavar2024generating,
  author = {Shreyas Bhat Brahmavar and Ashwin Srinivasan and Tirtharaj Dash and Sowmya Ramaswamy Krishnan and Lovekesh Vig and Arijit Roy and Raviprasad Aduri},
  title = {{Generating Novel Leads for Drug Discovery Using LLMs with Logical Feedback}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i1.27751},
}
Figure