Roughness of Molecular Property Landscapes and Its Impact on Modellability

Matteo Aldeghi, David E. Graff, Nathan C. Frey, Joseph A. Morrone, Edward O. Pyzer-Knapp, Kirk E. Jordan, Connor W. Coley

TL;DR

The authors introduce the Roughness Index (ROGI), a new quantitative metric that measures the "ruggedness" of molecular property landscapes. By analyzing how property dispersion changes as a dataset is coarse-grained, ROGI provides a robust predictor of how difficult a dataset will be for machine learning models to learn, showing a strong correlation with out-of-sample prediction errors.

What Problem Does This Paper Address?

In drug discovery, researchers often talk about "activity cliffs"—instances where a tiny structural change to a molecule leads to a massive jump in its biological activity. These cliffs create a "rough" landscape that is notoriously difficult for both human chemists to navigate and machine learning (ML) models to predict.

While the concept of landscape roughness is intuitive, quantifying it globally has been a challenge. Existing metrics are often local (only looking at pairs of molecules), require arbitrary hyperparameters, or are restricted to classification tasks. There hasn't been a universal, scalar value that tells a researcher: "This dataset is inherently 'jagged' and will likely result in high model error." The authors aim to fill this gap with a general measure of "modellability."

Methods

The core innovation is ROGI, which is loosely inspired by the concept of fractal dimension (like measuring the jaggedness of a coastline).

The calculation follows a "coarse-graining" logic:

  1. Clustering: The dataset is clustered using complete-linkage hierarchical clustering at increasing distance thresholds ($t$). At $t=0$, every molecule is its own cluster; at $t=1$, the entire dataset is one big cluster.
  2. Dispersion Loss: For each threshold, the authors calculate the weighted standard deviation ($\sigma_t$) of the property values within the clusters. As $t$ increases and clusters merge, the "fine-grained" details are lost, and the standard deviation of the cluster averages decreases.
  3. The Index: ROGI is defined as the area under the curve (AUC) of the "dispersion loss" ($2(\sigma_0 - \sigma_t)$) plotted against the distance threshold.

A high ROGI means property values change wildly even between very similar molecules (a rough landscape), while a low ROGI indicates that similar molecules have similar properties (a smooth landscape). Crucially, ROGI requires no tuning parameters once a molecular representation and distance metric are chosen.

Datasets

The authors validated ROGI across a diverse range of data:

  • Toy Surfaces: Six 2D analytical functions (F1–F6) ranging from perfectly smooth gradients to jagged checkerboards.
  • ZINC+GuacaMol: 13 regression datasets (2,000 molecules each) using "oracles" like LogP and QED.
  • Therapeutic Data Commons (TDC): 12 datasets covering pharmacokinetics and toxicology (e.g., Caco-2 permeability, Half-life).
  • ChEMBL: 30 curated Structure-Activity Relationship (SAR) datasets.
  • Classification: 50 binary classification datasets from TDC and Tox21.

Key Insights and Ideas

  • Roughness is Representation-Dependent: A landscape isn't just about the molecules; it's about how we describe them. The authors show that the same set of molecules can look "smooth" using one set of descriptors but "rough" using another (like Morgan fingerprints). ROGI captures this interaction.
  • Predicting the Unpredictable: The most striking "aha" moment is the strong linear correlation between ROGI and out-of-sample RMSE. If you calculate the ROGI of your training set, you can get a very good estimate of how poorly your model will perform on new data before you even train it.
  • Fractal Intuition works: By treating the property landscape as a geometric object and "zooming out" (coarse-graining), the authors successfully captured a global property that local measures like SALI (Structure-Activity Landscape Index) miss.

Key Results

  • Strong Correlation: Across 55 regression tasks, ROGI showed Pearson correlation coefficients ($r$) often exceeding 0.9 with the relative RMSE of various ML models (KNN, Random Forest, MLP, etc.).
  • Superiority over Existing Metrics: ROGI consistently outperformed existing indices like SARI and RMODI in anticipating model performance. For example, in many ChEMBL tasks where SARI struggled, ROGI maintained high correlation with model error.
  • Classification Utility: Although designed for regression, ROGI also correlates well with the accuracy of binary classifiers, particularly on balanced datasets.

Limitations and Future Work

  • Computational Complexity: The method relies on hierarchical clustering, which scales quadratically ($O(N^2)$) with the number of molecules. This makes it difficult to apply to datasets much larger than 10,000 molecules without subsampling.
  • Aleatoric Noise: ROGI assumes property values are exact. In reality, experimental noise can make a smooth landscape appear rough. The authors suggest that future versions could use modeled property values to "de-noise" the landscape.
  • Beyond Variance: The current ROGI uses the second moment (standard deviation) to measure dispersion. The authors note that higher-order moments or information entropy might be needed for more complex multi-class landscapes or optimization tasks.
  • Optimization Difficulty: While the paper links roughness to modellability (prediction), the authors acknowledge that linking it to optimization (finding the best molecule) is a more complex problem for future study.

Summary updated

Abstract

In molecular discovery and drug design, structure–property relationships and activity landscapes are often qualitatively or quantitatively analyzed to guide the navigation of chemical space. The roughness (or smoothness) of these molecular property landscapes is one of their most studied geometric attributes, as it can characterize the presence of activity cliffs, with rougher landscapes generally expected to pose tougher optimization challenges. Here, we introduce a general, quantitative measure for describing the roughness of molecular property landscapes. The proposed roughness index (ROGI) is loosely inspired by the concept of fractal dimension and strongly correlates with the out-of-sample error achieved by machine learning models on numerous regression tasks.
BibTeX
@article{aldeghi2022roughness,
  author = {Matteo Aldeghi and David E. Graff and Nathan C. Frey and Joseph A. Morrone and Edward O. Pyzer-Knapp and Kirk E. Jordan and Connor W. Coley},
  title = {{Roughness of Molecular Property Landscapes and Its Impact on Modellability}},
  journal = {Journal of Chemical Information and Modeling},
  year = {2022},
  doi = {10.1021/acs.jcim.2c00903},
}

Loading related content...

Figure