Are Scaling Laws of Universal MLIPs Universal?

Theo Keane, Anthony Onwuli, Richard Tomsett, Robert M. Forrest, Jonathan J. BeanarXiv 2026 · arXiv

TL;DR

Neural scaling laws for machine-learned interatomic potentials (MLIPs) are not as "universal" as previously thought. While more data generally improves performance, the rate of improvement depends heavily on the model architecture, the specific physical quantity being predicted (e.g., energy vs. force), and—most importantly—how well the training data's chemical distribution matches the intended test case.

What Problem Does This Paper Address?

In the world of Large Language Models (LLMs), "scaling laws" are a foundational concept: if you double the data and the parameters, you can reliably predict how much the error will drop. Researchers have been trying to apply this same logic to Machine-Learned Interatomic Potentials (MLIPs)—surrogate models that replace expensive quantum mechanical simulations (DFT) to predict how atoms interact.

However, MLIP data is fundamentally different from text. Generating a single data point (an atomic structure and its forces) requires a costly DFT calculation. Furthermore, the "chemical space" is vast and heterogeneous. This paper asks: Is the number of structures a sufficient variable for scaling? Does it matter which structures we pick? And do these scaling rules hold up when we test the model on a different distribution of materials?

Methods

The authors conducted a massive empirical study, training three representative MLIP architectures:

  • M3GNet: An invariant graph neural network.
  • MACE: A state-of-the-art equivariant model.
  • PET: An unconstrained architecture.

They compared 10 different data sampling policies to select training subsets ranging from 10,000 to 1.16 million structures. These policies included:

  • Random Sampling: The baseline.
  • Stratified Sampling: Ensuring a balance of chemical elements, formulas, or "protostructures."
  • Embedding-based Diversity: Using "Iterative DIRECT" and "Furthest-Point Sampling" (FPS) to pick structures that are most "different" from each other in the latent space of pretrained models.

Datasets

  • MPES-ALOE: A primary training pool of ~1.2 million structures created by combining the MatPES and MP-ALOE datasets.
  • In-distribution Test Set: A held-out 5% split of the MPES-ALOE dataset.
  • MP-r2SCAN: A distribution-shifted test set containing ~184k structures calculated with different DFT settings and a different chemical balance, used to test "out-of-distribution" (OOD) performance.

Key Insights and Ideas

  • Diversity $\neq$ Performance: Surprisingly, sophisticated embedding-based diversity sampling (like FPS) often performed worse than simple random sampling. Maximizing diversity in a model's latent space does not necessarily maximize the model's relevance to a specific prediction task.
  • The "Best" Policy is Context-Dependent: There is no single best way to pick data. A sampling policy that improves performance on a shifted test set (like chemical stratification) often degrades performance on the original in-distribution test set.
  • Chemical Divergence is the Best Predictor: The authors found a strong correlation between a model's relative loss and the Kullback–Leibler (KL) divergence between the training and test chemical distributions. If your training set doesn't "look" like your test set chemically, scaling laws won't save you.
  • Energy Scales Faster than Forces: Cohesive energy errors drop much more steeply with additional data than force or stress errors. This suggests that "energy-only" scaling studies might give an overly optimistic view of how quickly a model is actually improving for real-world molecular dynamics applications.
Figure 1: MPES-ALOE test set loss scaling laws of M3GNet, MACE and PET trained on data selected with 10 sampling policies. The loss function is defined in Eq. 1 and parameters are of a simple power law with asymptotic limit (see Eq. 2). A lower loss is better. The random policy is shown as thin gray lines in each panel, and the parameters are given in the upper left panel.

Key Results

  • Scaling Exponents ($\alpha$): The rate of error decay varied by 1.6–2× between different sampling policies within the same architecture.
  • Architecture Sensitivity: While MACE (equivariant) generally outperforms M3GNet (invariant), the gap narrows or even reverses at small data scales (e.g., 10k structures) depending on the sampling policy used.
  • Predictive Correlation: Spearman correlations of 0.68–0.89 were observed between model loss and the size-adjusted KL divergence of chemical space, confirming that "relevance" to the test set is a dominant factor in performance.
Comparison of PET model scaling on in-distribution (MPES-ALOE) vs. shifted (MP-r2SCAN) test sets. Policy rankings flip depending on the evaluation task.

Limitations and Future Work

  • Limited Architectures: The study focused on three specific model configurations. While representative, they don't cover the entire "zoo" of MLIPs.
  • Data Range: The training sizes spanned only two orders of magnitude (10k to 1M). Scaling behavior might change at the "extreme" ends (e.g., billions of structures).
  • Controlled OOD: The distribution shift in MP-r2SCAN involves both chemical changes and different DFT settings simultaneously, making it hard to isolate exactly which factor causes the performance drop.
  • Future Direction: The authors suggest that future scaling reports should always specify the sampling policy and the intended use case, rather than just reporting structure counts. Investigating how magnetic degrees of freedom affect scaling is also a noted area for future research.

Summary updated

Abstract

Neural scaling laws enable the prediction of returns from larger training datasets, but machine-learned interatomic potential (MLIP) datasets are heterogeneous and expensive to generate, and equal numbers of structures need not contain equal amounts of useful information. We study data scaling under fixed model configurations for M3GNet, MACE, and PET trained on subsets of a combined MatPES-MP-ALOE training pool. Across training structure subsets from 10,000 to 640,000, together with the full 1.16-million-structure training set, we compare uniform random sampling, three chemistry- and structure-informed stratification policies, iterative DIRECT, and furthest-point sampling using embeddings from three pretrained MLIPs. Models are evaluated using cohesive-energy, force, stress, and aggregate losses on both a held-out test set from the parent distribution and the distribution shifted MP-r$^2$SCAN test set. We observe power-law scaling improvements with increasing data, but the fitted scaling exponent, apparent error floor, and relative sample efficiency depend on the model architecture, target quantity, data sampling policy, and test set distribution. Sampling-policy rankings can reverse across test distributions: chemical stratification can improve cross-dataset performance while degrading in-distribution performance, whereas the embedding-space diversity policies frequently under-perform random sampling. Relative loss is strongly associated with test-to-train chemical-space divergence. These results show that structure count is a necessary but insufficient scaling variable for MLIPs. For training MLIPs, the return on additional first-principles calculations depends not only on how many additional structures are included, but also on how their chemical and configurational content aligns with the model's intended use.
BibTeX
@misc{keane2026are,
  author = {Theo Keane and Anthony Onwuli and Richard Tomsett and Robert M. Forrest and Jonathan J. Bean},
  title = {{Are Scaling Laws of Universal MLIPs Universal?}},
  year = {2026},
  eprint = {2611.15603},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15603},
}
Figure