Extrapolating Machine Learning Models to Novel Chemical Spaces

85 concepts67 papers1 proposal

Machine learning models are exceptionally good at finding patterns within the boundaries of their training data. However, scientific discovery is fundamentally an act of exploration, not interpolation. If we want to discover a breakthrough catalyst, a high-efficiency thermoelectric material, or a novel drug scaffold, we are looking for global optima that lie far outside known distributions.

As Tomer Weiss et al. observe in Guided diffusion for inverse molecular design, most generative models are restricted to interpolating within the range of their training labels. This creates an "extrapolation gap" where models fail to predict or generate properties that exceed the best examples they have already seen, a limitation echoed by Alexia Jolicoeur-Martineau et al. in Any-Property-Conditional Molecule Generation with Self-Criticism using Spanning Trees. To make machine learning a reliable engine for discovery, we must understand why models fail when they leave familiar territory and how we can design architectures to cross this gap.

The Dimensions of Out-of-Distribution Failure

Why exactly do models break down when they leave familiar territory? The literature identifies two primary failure modes: structural shifts and configurational shifts.

Structural and Compositional Shifts

Models trained on specific chemical families or crystal symmetries struggle when encountering entirely new regions of the periodic table. John A. Keith et al. note in Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems that models frequently fail when encountering novel bonding types. Similarly, Raymundo Arroyave argues in Phase Stability Through Machine Learning that the inability to handle out-of-distribution chemical spaces severely limits the discovery of novel materials. When a model encounters an unrepresented element combination or an unfamiliar crystal symmetry, its internal representations lack the necessary context to map these structures to accurate properties.

Configurational and Geometric Shifts

Interatomic potentials and molecular dynamics models are often trained on near-equilibrium bulk structures. When a simulation explores highly distorted geometries, point defects, or high-energy transition states, the underlying physics captured by the model breaks down. Bowen Deng et al. document this in Systematic softening in universal machine learning interatomic potentials, showing that universal potentials systematically underpredict forces in out-of-distribution environments. This matches findings by Silvan Käser et al. in Machine Learning Models of Vibrating H2CO: Comparing Reproducing Kernels, FCHL, and PhysNet, where models lose accuracy on highly distorted molecular geometries because the training data over-samples stable, low-energy configurations.

Why Continuous Latent Spaces Enforce Interpolation

To understand why generative AI struggles to discover truly novel motifs, we must look at how they represent data. Modern deep learning models typically map discrete molecular structures into a continuous latent space. Optimization then occurs by moving along this continuous manifold.

However, as Benjamin Kurt Miller et al. explain in FlowMM: Generating Materials with Riemannian Flow Matching, this design forces the model to interpolate within the training manifold, making it blind to entirely novel structural motifs. When a model samples from a continuous latent space, it naturally blends features of known molecules.

To break free from this geometric constraint, Minghao Guo proposes an alternative in Data-Efficient Graph Grammar Learning for Molecular Generation: instead of continuous latent spaces, models can learn discrete "graph grammars"—production rules that assemble molecules from substructures. By optimizing these discrete assembly rules rather than moving through a continuous space, models can extrapolate to innovative architectures even when trained on highly scarce data.

Benchmarking the Extrapolation Gap

Before we can fix the extrapolation gap, we must be able to measure it. Traditional evaluation metrics evaluate models on random splits of a dataset, which primarily tests interpolation.

To address this, Evan R. Antoniuk et al. introduced BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models, demonstrating that even advanced chemical foundation models experience a tripling of error rates when forced to extrapolate.

To evaluate generative models rather than just predictive ones, Mai Elkady et al. developed a framework called Vertical Validation in Vertical Validation: Evaluating Implicit Generative Models for Graphs on Thin Support Regions. By systematically thinning out specific regions of the training data, they test whether a model can extrapolate into the "thin support" regions where true discovery happens, providing a metric for a model's capacity to generate rare or unprecedented structures.

Strategies for True Extrapolation

How do we build models capable of navigating out-of-distribution chemical space? Researchers are exploring several structural and algorithmic interventions:

Key Foundational Work

Can machine learning identify the next high-temperature superconductor? Examining extrapolation performance for materials discovery (2018) — contributed 3 concept elements to this direction.
Hybrid Generative-Discriminative Models for Inverse Materials Design (2018) — contributed 2 concept elements to this direction.
Strategies and Software for Machine Learning Accelerated Discovery in Transition Metal Chemistry (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_out_of_distribution_generalization_chemi_249923e0,
  title = {out-of-distribution generalization (chemical space extrapolation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/out-of-distribution-generalization-chemical-space-extrapolation/rdr_249923e0694e4afe8cea72c4ce05d031},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_out_of_distribution_generalization_chemi_249923e0_v1,
  title = {out-of-distribution generalization (chemical space extrapolation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/out-of-distribution-generalization-chemical-space-extrapolation/rdr_249923e0694e4afe8cea72c4ce05d031/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake