Abstract
Molecular docking is a critical component of computer-aided drug discovery, yet current machine learning (ML) models often fail to generalize to novel protein structures. This paper introduces DOCKGEN, a rigorous benchmark based on structural domain splits, and proposes strategies to bridge the generalization gap: model scaling and Confidence Bootstrapping. By leveraging a discriminative confidence model to provide pseudo-labels for a generative diffusion model, the authors demonstrate significant performance improvements on unseen protein pockets, outperforming traditional search-based methods.
Introduction
Predicting the binding pose of a small molecule within a protein's active site—molecular docking—is essential for identifying potential drug candidates. While traditional physics-based search methods are robust, they are computationally intensive. Recent ML-based "blind docking" models, such as DiffDock, offer orders-of-magnitude speedups by treating docking as a generative task, finding the pocket and the pose simultaneously.
However, the evaluation of these models has been historically flawed. Most benchmarks split data based on sequence similarity or time. Because evolutionarily distinct proteins often share conserved structural folds and binding pockets, models can achieve high scores by memorizing known pocket geometries rather than learning the fundamental physics of molecular interactions. When these models encounter truly novel protein domains, their performance often collapses. This paper aims to quantify this "generalization gap" and provides a framework to overcome it.
The DOCKGEN Benchmark: A True Test of Generalization
To address the limitations of existing benchmarks, the authors developed DOCKGEN. Unlike previous datasets that rely on sequence similarity, DOCKGEN partitions data based on the Evolutionary Classification of Protein Domains (ECOD). This ensures that the test set contains structural folds entirely absent from the training set.
The results on DOCKGEN confirm the severity of the generalization gap. Standard ML models that perform well on sequence-split benchmarks struggle significantly; for instance, the original DiffDock model achieved only a 7.1% success rate. This benchmark provides a realistic assessment of how docking models will perform in the "dark matter" of the proteome—proteins with no known structural homologs.
Scaling and Synthetic Data Augmentation
The authors first investigated whether the "Bitter Lesson" of AI—that more data and larger models eventually lead to better performance—applies to docking generalization. They attacked the problem from two angles:
- DiffDock-L: They scaled the architecture to approximately 30 million parameters (up from 20M) and expanded the training set using the Binding MOAD collection.
- Synthetic Sidechain Training: To increase pocket diversity, they employed a "van der Mer" inspired strategy. By treating protein sidechains as "fake ligands," the model learns to predict their positions relative to the protein backbone. This provides a nearly inexhaustible source of diverse, albeit simplified, binding examples, helping the model learn general geometric constraints of protein-ligand interactions.
Confidence Bootstrapping: Self-Training for New Pockets
The most significant contribution of the paper is Confidence Bootstrapping, a self-training strategy designed for proteins where no ground-truth ligand pose is available. This method exploits a key insight: discriminative models (confidence scorers) generalize to new domains much better than generative models.
The bootstrapping loop consists of three phases:
- Rollout: The diffusion model generates multiple candidate poses for a ligand in a novel pocket.
- Score: A separate confidence model ranks these poses, identifying those most likely to be correct.
- Update: The diffusion model is fine-tuned using these high-confidence poses as "pseudo-labels."
Crucially, the authors found that fine-tuning the early, coarse-grained steps of the diffusion process is more effective than focusing on the final refinement. These early steps determine the global placement of the ligand (which pocket it enters), which is the primary failure mode when generalizing to new structures.
Key Results and Insights
The combination of scaling and bootstrapping yields substantial gains over previous state-of-the-art methods:
- DiffDock-L reached a 22.6% success rate on DOCKGEN, surpassing the best traditional search-based methods, which typically peak around 17.5%.
- Confidence Bootstrapping demonstrated remarkable efficacy on specific unseen protein clusters, improving success rates from 9.8% to 24.0%. In half of the tested domains, the success rate exceeded 30% after fine-tuning.
- The Intelligence Gap: The success of bootstrapping confirms that it is easier for a model to recognize a correct pose than to generate one from scratch in an unfamiliar environment.
Limitations and Future Work
While promising, the approach has limitations. The Confidence Bootstrapping process requires approximately 8 hours of GPU time per protein cluster, which may be a hurdle for some applications. Additionally, the synthetic data uses amino acids as ligands, which do not fully capture the chemical complexity of drug-like small molecules. Future work will likely focus on incorporating affinity information and further optimizing the computational efficiency of the self-training loop.
Conclusion
The DOCKGEN benchmark reveals that structural generalization remains a formidable challenge for ML in drug discovery. However, by combining model scaling with "Confidence Bootstrapping," it is possible to teach models to navigate unseen protein pockets. These strategies provide a roadmap for developing more robust models capable of exploring the vast, uncharted regions of the human proteome.
Summary updated