Roughness of molecular property landscapes and its impact on modellability

Matteo Aldeghi, David Graff, Nathan C. Frey, Joseph A. Morrone, Edward O. Pyzer‐Knapp, Kirk E. Jordan, Connor W. Coley

Abstract

In molecular discovery and drug design, structure-property relationships and activity landscapes are often qualitatively or quantitatively analyzed to guide the navigation of chemical space. The roughness (or smoothness) of these molecular property landscapes is one of their most studied geometric attributes, as it can characterize the presence of activity cliffs, with rougher landscapes generally expected to pose tougher optimization challenges. Here, we introduce a general, quantitative measure for describing the roughness of molecular property landscapes. The proposed roughness index (ROGI) is loosely inspired by the concept of fractal dimension and strongly correlates with the out-of-sample error achieved by machine learning models on numerous regression tasks.

Introduction

The concept of "activity cliffs"—where minor structural modifications lead to significant changes in biological activity—is central to drug discovery. These cliffs create "rough" landscapes that are challenging for both human intuition and machine learning (ML) algorithms. While the notion of landscape roughness is widely recognized, a universal, quantitative metric to measure it globally has remained elusive. Existing metrics are often local, hyperparameter-dependent, or limited to classification tasks.

To address this, we introduce the Roughness Index (ROGI). ROGI provides a scalar value that characterizes the inherent "modellability" of a dataset by quantifying its global ruggedness. It serves as a robust predictor of the difficulty an ML model will face when learning from a specific molecular property landscape.

Methodology: The Roughness Index (ROGI)

ROGI is inspired by fractal dimension analysis, specifically how the perceived "jaggedness" of an object changes as one "zooms out" or coarse-grains the data. The calculation involves three primary steps:

  1. Hierarchical Clustering: The dataset is organized using complete-linkage hierarchical clustering across a range of distance thresholds ($t \in [0, 1]$). At $t=0$, each molecule is an individual cluster; as $t$ increases, clusters merge until the entire dataset forms a single cluster at $t=1$.
Visualization of the coarse-graining process via hierarchical clustering at a distance threshold $t=0.75$
  1. Dispersion Loss Calculation: For every threshold $t$, the weighted standard deviation ($\sigma_t$) of property values within the clusters is calculated. As the landscape is coarse-grained (increasing $t$), fine-grained property variations are smoothed out, and the internal dispersion of cluster averages decreases.
  2. ROGI Definition: ROGI is defined as the area under the curve (AUC) of the "dispersion loss," calculated as $2(\sigma_0 - \sigma_t)$, plotted against the distance threshold $t$.
The Roughness Index (ROGI) defined as the area under the curve (AUC) of dispersion loss

A high ROGI indicates a rough landscape where properties fluctuate significantly between similar molecules, whereas a low ROGI signifies a smooth landscape where structural similarity implies property similarity.

Experimental Validation and Results

The utility of ROGI was evaluated across a broad spectrum of data, including toy analytical surfaces, regression datasets from ZINC and GuacaMol, pharmacokinetics and toxicology data from the Therapeutic Data Commons (TDC), and curated SAR datasets from ChEMBL.

Key Insights

  • Representation Dependence: Landscape roughness is not an intrinsic property of the molecules alone but depends on the chosen molecular representation. ROGI effectively captures how different descriptors (e.g., Morgan fingerprints vs. physical properties) alter the perceived smoothness of the same dataset.
  • Predicting Model Performance: A significant finding is the strong linear correlation between ROGI and out-of-sample prediction error. Across 55 regression tasks, ROGI achieved Pearson correlation coefficients ($r$) often exceeding 0.9 with the relative RMSE of various ML models, including Random Forests and MLPs.
  • Superiority over Existing Metrics: ROGI consistently outperformed traditional indices like SARI and RMODI in predicting model performance, particularly in complex ChEMBL tasks.

Discussion and Future Directions

While ROGI provides a powerful tool for assessing dataset modellability, it has limitations. The reliance on hierarchical clustering introduces $O(N^2)$ computational complexity, necessitating subsampling for very large datasets. Additionally, experimental noise (aleatoric uncertainty) can artificially inflate the measured roughness. Future work may involve "de-noising" landscapes using modeled values or extending the index to multi-class and optimization-specific tasks.

By linking geometric roughness to predictive accuracy, ROGI offers a principled way to evaluate and select molecular representations and to anticipate the challenges of specific drug discovery tasks before model training begins.

Summary updated

Abstract

In molecular discovery and drug design, structure-property relationships and activity landscapes are often qualitatively or quantitatively analyzed to guide the navigation of chemical space. The roughness (or smoothness) of these molecular property landscapes is one of their most studied geometric attributes, as it can characterize the presence of activity cliffs, with rougher landscapes generally expected to pose tougher optimization challenges. Here, we introduce a general, quantitative measure for describing the roughness of molecular property landscapes. The proposed roughness index (ROGI) is loosely inspired by the concept of fractal dimension and strongly correlates with the out-of-sample error achieved by machine learning models on numerous regression tasks.
BibTeX
@article{aldeghi2022roughness,
  author = {Matteo Aldeghi and David Graff and Nathan C. Frey and Joseph A. Morrone and Edward O. Pyzer‐Knapp and Kirk E. Jordan and Connor W. Coley},
  title = {{Roughness of molecular property landscapes and its impact on modellability}},
  journal = {Journal of Chemical Information and Modeling},
  year = {2022},
  doi = {10.1021/acs.jcim.2c00903},
}
Figure