Deep Confident Steps to New Pockets: Strategies for Docking Generalization

Gabriele Corso, Arthur Deng, Benjamin Fry, Nicholas Polizzi, Regina Barzilay, Tommi S. Jaakkola

Abstract

Molecular docking is a critical component of computer-aided drug discovery, yet current machine learning (ML) models often fail to generalize to novel protein structures. This paper introduces DOCKGEN, a rigorous benchmark based on structural domain splits, and proposes strategies to bridge the generalization gap: model scaling and Confidence Bootstrapping. By leveraging a discriminative confidence model to provide pseudo-labels for a generative diffusion model, the authors demonstrate significant performance improvements on unseen protein pockets, outperforming traditional search-based methods.

Introduction

Predicting the binding pose of a small molecule within a protein's active site—molecular docking—is essential for identifying potential drug candidates. While traditional physics-based search methods are robust, they are computationally intensive. Recent ML-based "blind docking" models, such as DiffDock, offer orders-of-magnitude speedups by treating docking as a generative task, finding the pocket and the pose simultaneously.

However, the evaluation of these models has been historically flawed. Most benchmarks split data based on sequence similarity or time. Because evolutionarily distinct proteins often share conserved structural folds and binding pockets, models can achieve high scores by memorizing known pocket geometries rather than learning the fundamental physics of molecular interactions. When these models encounter truly novel protein domains, their performance often collapses. This paper aims to quantify this "generalization gap" and provides a framework to overcome it.

The DOCKGEN Benchmark: A True Test of Generalization

To address the limitations of existing benchmarks, the authors developed DOCKGEN. Unlike previous datasets that rely on sequence similarity, DOCKGEN partitions data based on the Evolutionary Classification of Protein Domains (ECOD). This ensures that the test set contains structural folds entirely absent from the training set.

The results on DOCKGEN confirm the severity of the generalization gap. Standard ML models that perform well on sequence-split benchmarks struggle significantly; for instance, the original DiffDock model achieved only a 7.1% success rate. This benchmark provides a realistic assessment of how docking models will perform in the "dark matter" of the proteome—proteins with no known structural homologs.

Scaling and Synthetic Data Augmentation

The authors first investigated whether the "Bitter Lesson" of AI—that more data and larger models eventually lead to better performance—applies to docking generalization. They attacked the problem from two angles:

  1. DiffDock-L: They scaled the architecture to approximately 30 million parameters (up from 20M) and expanded the training set using the Binding MOAD collection.
  2. Synthetic Sidechain Training: To increase pocket diversity, they employed a "van der Mer" inspired strategy. By treating protein sidechains as "fake ligands," the model learns to predict their positions relative to the protein backbone. This provides a nearly inexhaustible source of diverse, albeit simplified, binding examples, helping the model learn general geometric constraints of protein-ligand interactions.

Confidence Bootstrapping: Self-Training for New Pockets

The most significant contribution of the paper is Confidence Bootstrapping, a self-training strategy designed for proteins where no ground-truth ligand pose is available. This method exploits a key insight: discriminative models (confidence scorers) generalize to new domains much better than generative models.

The bootstrapping loop consists of three phases:

  • Rollout: The diffusion model generates multiple candidate poses for a ligand in a novel pocket.
  • Score: A separate confidence model ranks these poses, identifying those most likely to be correct.
  • Update: The diffusion model is fine-tuned using these high-confidence poses as "pseudo-labels."

Crucially, the authors found that fine-tuning the early, coarse-grained steps of the diffusion process is more effective than focusing on the final refinement. These early steps determine the global placement of the ligand (which pocket it enters), which is the primary failure mode when generalizing to new structures.

Key Results and Insights

The combination of scaling and bootstrapping yields substantial gains over previous state-of-the-art methods:

  • DiffDock-L reached a 22.6% success rate on DOCKGEN, surpassing the best traditional search-based methods, which typically peak around 17.5%.
  • Confidence Bootstrapping demonstrated remarkable efficacy on specific unseen protein clusters, improving success rates from 9.8% to 24.0%. In half of the tested domains, the success rate exceeded 30% after fine-tuning.
  • The Intelligence Gap: The success of bootstrapping confirms that it is easier for a model to recognize a correct pose than to generate one from scratch in an unfamiliar environment.

Limitations and Future Work

While promising, the approach has limitations. The Confidence Bootstrapping process requires approximately 8 hours of GPU time per protein cluster, which may be a hurdle for some applications. Additionally, the synthetic data uses amino acids as ligands, which do not fully capture the chemical complexity of drug-like small molecules. Future work will likely focus on incorporating affinity information and further optimizing the computational efficiency of the self-training loop.

Conclusion

The DOCKGEN benchmark reveals that structural generalization remains a formidable challenge for ML in drug discovery. However, by combining model scaling with "Confidence Bootstrapping," it is possible to teach models to navigate unseen protein pockets. These strategies provide a roadmap for developing more robust models capable of exploring the vast, uncharted regions of the human proteome.

Summary updated

BibTeX
@inproceedings{corso2024deep,
  author = {Gabriele Corso and Arthur Deng and Benjamin Fry and Nicholas Polizzi and Regina Barzilay and Tommi S. Jaakkola},
  title = {{Deep Confident Steps to New Pockets: Strategies for Docking Generalization}},
  booktitle = {International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=UfBIxpTK10},
}
Figure