Abstract
Structure-based drug design (SBDD) is crucial for developing specific and effective therapeutics against protein targets but remains challenging due to complex protein-ligand interactions and vast chemical space. Although language models (LMs) have excelled in natural language processing, their application in SBDD is underexplored. To bridge this gap, we introduce a method, known as Frag2Seq, to apply LMs to SBDD by generating molecules in a fragment-based manner in which fragments correspond to functional modules. We transform 3D molecules into fragment-informed sequences using SE(3)-equivariant molecule and fragment local frames, extracting SE(3)-invariant sequences that preserve geometric information of 3D fragments. Furthermore, we incorporate protein pocket embeddings obtained from a pre-trained inverse folding model into the LMs via cross-attention to capture protein-ligand interaction, enabling effective target-aware molecule generation. Benefiting from employing LMs with fragment-based generation and effective protein context encoding, our model achieves the best performance on binding vina score and chemical properties such as QED and Lipinski, which shows our model's efficacy in generating drug-like ligands with higher binding affinity against target proteins. Moreover, our method also exhibits higher sampling efficiency compared to atom-based autoregressive and diffusion baselines with at most ~300x speedup.
Introduction
Structure-based drug design (SBDD) involves creating molecules that fit precisely into a specific protein "pocket" to modulate biological functions. This task is notoriously difficult due to the vastness of chemical space and the intricate physics of protein-ligand interactions. While Language Models (LMs) have revolutionized fields like natural language processing, their application to 3D molecular structures has been limited by the difficulty of representing 3D geometry as 1D sequences.
Current machine learning approaches typically rely on atom-by-atom autoregressive generation or computationally intensive diffusion models. Frag2Seq introduces a paradigm shift by tokenizing molecules into geometric fragments, enabling powerful LMs to generate high-affinity, drug-like ligands with significant efficiency gains.
Methodology
The Frag2Seq framework bridges the gap between 3D geometry and 1D sequences through three core innovations:
3D Molecule Fragmentation
Instead of treating a molecule as a collection of individual atoms, Frag2Seq decomposes it into functional fragments (such as rings or hydroxy groups) by identifying and cutting rotatable bonds. This approach ensures that the "building blocks" of the generated molecules are chemically stable functional modules from the outset.
Geometry-Aware Tokenization
To make 3D data readable for an LM, the model constructs $SE(3)$-equivariant local frames for each fragment. This allows for the extraction of $SE(3)$-invariant features—specifically spherical coordinates $(d, \theta, \phi)$ and rotation vectors. Because these features are invariant, the model does not need to learn that a molecule is identical regardless of its orientation or position in space, greatly simplifying the learning task.
Target-Aware Generation
Frag2Seq incorporates protein context using ESM-IF1, a pre-trained inverse folding model. These protein embeddings are integrated into a GPT-style architecture via cross-attention, allowing the model to dynamically "look" at the protein pocket while determining the next fragment to place.
Key Results and Discussion
The researchers evaluated Frag2Seq using the CrossDocked2020 dataset, a standard benchmark in SBDD.
- Binding Affinity and Drug-likeness: Frag2Seq outperformed several strong baselines, including diffusion models (DiffSBDD) and atom-based models (Pocket2Mol). It achieved a superior Vina Score (-7.366) and scored highest on QED (0.645) and Lipinski (4.989) metrics, indicating the generation of viable drug candidates with high binding affinity.
- Sampling Efficiency: By generating molecules fragment-by-fragment, the model significantly reduces the number of steps required. Frag2Seq generates approximately 2.0 samples per second, representing a 200x to 300x speedup over traditional diffusion-based methods.
- Coordinate Systems: The authors found that spherical coordinates are particularly well-suited for LMs because they are bounded within a smaller numerical range, minimizing information loss during the quantization process required for tokenization.
Conclusion
Frag2Seq demonstrates that fragment-based generation combined with geometry-aware tokenization allows language models to be highly effective in structure-based drug design. While future work is needed to address quantization loss and automate final bond inference, the model's ability to generate high-quality ligands with unprecedented speed marks a major step forward in computational drug discovery.
Summary updated