Bento: Benchmarking Classical and AI Docking on Drug Design–Relevant Data

Marina A. Pak, Daria Frolova, Sergei A. Nikolenko, Talgat Daulbaev, Daria Ryabchenko, Anna Litvin, Pavel E. Gurevich, Kamil Garifullin, Alexander V. Shapeev, Ivan Oseledets, Dmitry N. Ivankov

TL;DR

This paper introduces BENTO, a comprehensive benchmark for evaluating protein-ligand interaction prediction tools, including classical, deep learning, and co-folding methods. It reveals that while current tools are useful for drug design, they struggle to generalize to novel protein pockets, with deep learning models being particularly prone to overfitting.

What Problem Does This Paper Address?

Predicting how small molecules (ligands) bind to proteins is a cornerstone of drug discovery. This process, known as docking, helps researchers identify potential drug candidates. While classical physics-based methods have been the standard, recent advances in Artificial Intelligence (AI), particularly deep learning and co-folding approaches (like AlphaFold), have shown promise. However, it's unclear how these new AI methods truly stack up against established techniques, especially when dealing with the diverse and complex data encountered in real-world drug design. Existing benchmarks often focus on generalization to new proteins but don't always account for the nuances of ligand complexity or the specific structural features of protein binding pockets, leading to potentially misleading performance assessments. This paper aims to provide a more rigorous and nuanced evaluation by introducing BENTO, a benchmark designed to disentangle these factors and assess performance on drug-design-relevant scenarios.

Methods

The core of this work is the BENTO benchmark, a systematic evaluation of 11 protein-ligand interaction prediction tools. These tools span three categories:

  • Classical physics-based docking: AutoDock Vina and smina.
  • Deep learning (DL)-based docking: Gnina, DiffDock, Uni-Mol Docking V2, NeuralPLexer, FlowDock, and Matcha.
  • Co-folding approaches: AlphaFold3, Boltz-2, and Chai-1.

BENTO goes beyond simply testing on standard datasets. It introduces curated subsets that stratify data based on:

  • Ligand classes: Grouping ligands by chemical type (e.g., amino acids, nucleosides, lipids).
  • Ligand complexity: Categorizing ligands by size (number of heavy atoms), flexibility (number of rotatable bonds), and burial within the protein pocket.
  • Drug-like properties: Focusing on ligands that resemble those found in typical drug discovery screening libraries.
  • Protein pocket and ligand similarity: Quantifying how similar the protein binding pockets and the ligands are to those in the training sets, using metrics like G-LoSA (for pockets) and Tanimoto similarity (for ligands).

This detailed stratification allows for a more precise understanding of how different factors influence tool performance and helps identify specific weaknesses, such as generalization failures that might be masked in broader benchmarks.

Datasets

BENTO utilizes established datasets for evaluation:

  • PDBBind Timesplit test: A dataset designed to assess generalization over time.
  • PoseBusters: Known for its high-quality data and focus on physical validity.
  • Astex: A dataset often used for its relevance to drug candidate design.
  • DockGen: Specifically designed to include proteins with dissimilar binding pockets.

In addition to these, BENTO creates derived subsets from these datasets to specifically test performance on drug-like ligands, similar/dissimilar protein pockets, and similar/dissimilar ligands. The paper also notes the training sets used for the evaluated tools, which primarily include PDBBind and Binding MOAD, with some tools trained on custom datasets.

Key Insights and Ideas

  • Dataset Curation is Crucial: The study strongly emphasizes that how you curate your test data significantly impacts results. Simply using standard datasets can obscure critical failures, especially regarding generalization to novel protein pockets. Filtering by pocket structural similarity and controlling ligand complexity are essential.
  • DL vs. Classical for Drug Design: For typical drug-like ligands and proteins similar to training data, DL-based docking and classical methods perform comparably. This means classical methods, which are often faster, remain highly relevant for virtual screening.
  • Co-folding Shines on Complexity: When dealing with structurally complex ligands (e.g., macrocycles, cofactors), co-folding tools generally outperform other methods.
  • Generalization to Unseen Pockets is a Major Hurdle: All methods struggle to generalize to protein pockets that are structurally different from their training data. Deep learning models, in particular, show a strong tendency to overfit, meaning they perform well on data similar to what they've seen but poorly on truly novel cases.
  • Physics-Informed AI is Promising: The paper suggests that incorporating physics-based principles into deep learning frameworks is key to improving robustness and generalization.

Key Results

  • Astex Dataset: Showed the highest performance overall, likely due to its abundance of "regular" small molecules and similarity to training data.
  • DockGen Dataset: Proved to be the most challenging, with co-folding methods like AlphaFold3 and Chai-1 performing best but still achieving only modest success rates (around 14% valid predictions).
  • Drug-like Ligands: Gnina achieved the highest fraction of physically valid predictions (70%) on a drug-like subset from PoseBusters, with smina also outperforming co-folding and purely DL methods in this specific scenario.
  • Unseen Pockets: Performance dropped significantly across nearly all methods when tested on dissimilar protein pockets, with DL approaches showing the most pronounced decline. Co-folding and physics-based methods performed similarly poorly in this challenging scenario.
  • Ligand Complexity: Co-folding tools generally excelled on complex ligands, while regular small molecules were predicted with comparable accuracy across most methods.

Limitations and Future Work

Limitations: * Overfitting in DL: A significant limitation highlighted is the tendency of DL models to overfit, restricting their generalizability to unseen protein pockets. * Uni-Mol Docking V2 Comparison: The paper notes that Uni-Mol Docking V2's "pocket-aware" mode, which rigidly crops the pocket, might leak information and make direct comparison with other tools challenging. They used predicted pocket centers for Uni-Mol to ensure fairness. * Dataset Specificity: While BENTO is comprehensive, the performance of tools can still be dataset-dependent, and the study focuses on specific types of protein-ligand interactions.

Future Work: * Improving Generalization: The study points to the need for developing more robust protein-ligand prediction models that can generalize better to unseen pockets. * Physics-Informed DL: The authors advocate for integrating physics-based principles into DL architectures to enhance predictive robustness and generalization. * Rigorous Benchmarking: BENTO itself serves as a framework for future research, providing a rigorous and transparent way to diagnose limitations and guide the development of better models.

Summary updated

Abstract

A bstract Recent advances in artificial intelligence have introduced deep learning and co-folding approaches for predicting protein-ligand complexes, raising the question of their applicability and how they compare with classical docking methods. In this work, we present a thorough benchmarking study of eleven tools for protein-ligand interaction prediction, spanning classical molecular docking methods, deep learning-based models, and co-folding algorithms. While most related benchmarking efforts primarily assess the generalization capacity, we extend the analysis to also evaluate the performance on drug design-relevant data and across different classes of protein-ligand complexes. Here, we introduce B ento , a comprehensive benchmark that evaluates 11 tools for protein-ligand interaction prediction – both established and recently developed – across four test datasets and multiple derived subsets in a pocket-aware setup. We show that 1) careful dataset curation is essential – filtering by pocket structural similarity and controlling ligand complexity exposes generalization failures that are obscured in conventional benchmarks; 2) classical and deep learning-based docking tools perform similarly well on drug-like ligands, making them comparably useful for virtual screening, with physics-based methods offering a clear advantage in speed; 3) co-folding tools outperform other approaches on structurally complex ligands, whereas most methods achieve similar accuracy on regular small molecules; and 4) all methods struggle to generalize to unseen pockets, with deep learning models being the most prone to overfitting. Overall, our results show that while current docking and DL-based approaches are reliable for many drug-design-relevant scenarios, genuine pocket-level generalization remains an open challenge. B ento provides a rigorous and transparent framework for diagnosing these limitations and guiding the development of more robust protein-ligand prediction models. The data and code of Bento are available at https://github.com/LigandPro/Bento .
BibTeX
@misc{pak2026bento,
  author = {Marina A. Pak and Daria Frolova and Sergei A. Nikolenko and Talgat Daulbaev and Daria Ryabchenko and Anna Litvin and Pavel E. Gurevich and Kamil Garifullin and Alexander V. Shapeev and Ivan Oseledets and Dmitry N. Ivankov},
  title = {{Bento: Benchmarking Classical and AI Docking on Drug Design–Relevant Data}},
  year = {2026},
  url = {https://openreview.net/forum?id=kIxAQxUZHq},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2026},
}
Figure