TL;DR
Neural scaling laws for machine-learned interatomic potentials (MLIPs) are not as "universal" as previously thought. While more data generally improves performance, the rate of improvement depends heavily on the model architecture, the specific physical quantity being predicted (e.g., energy vs. force), and—most importantly—how well the training data's chemical distribution matches the intended test case.
What Problem Does This Paper Address?
In the world of Large Language Models (LLMs), "scaling laws" are a foundational concept: if you double the data and the parameters, you can reliably predict how much the error will drop. Researchers have been trying to apply this same logic to Machine-Learned Interatomic Potentials (MLIPs)—surrogate models that replace expensive quantum mechanical simulations (DFT) to predict how atoms interact.
However, MLIP data is fundamentally different from text. Generating a single data point (an atomic structure and its forces) requires a costly DFT calculation. Furthermore, the "chemical space" is vast and heterogeneous. This paper asks: Is the number of structures a sufficient variable for scaling? Does it matter which structures we pick? And do these scaling rules hold up when we test the model on a different distribution of materials?
Methods
The authors conducted a massive empirical study, training three representative MLIP architectures:
- M3GNet: An invariant graph neural network.
- MACE: A state-of-the-art equivariant model.
- PET: An unconstrained architecture.
They compared 10 different data sampling policies to select training subsets ranging from 10,000 to 1.16 million structures. These policies included:
- Random Sampling: The baseline.
- Stratified Sampling: Ensuring a balance of chemical elements, formulas, or "protostructures."
- Embedding-based Diversity: Using "Iterative DIRECT" and "Furthest-Point Sampling" (FPS) to pick structures that are most "different" from each other in the latent space of pretrained models.
Datasets
- MPES-ALOE: A primary training pool of ~1.2 million structures created by combining the MatPES and MP-ALOE datasets.
- In-distribution Test Set: A held-out 5% split of the MPES-ALOE dataset.
- MP-r2SCAN: A distribution-shifted test set containing ~184k structures calculated with different DFT settings and a different chemical balance, used to test "out-of-distribution" (OOD) performance.
Key Insights and Ideas
- Diversity $\neq$ Performance: Surprisingly, sophisticated embedding-based diversity sampling (like FPS) often performed worse than simple random sampling. Maximizing diversity in a model's latent space does not necessarily maximize the model's relevance to a specific prediction task.
- The "Best" Policy is Context-Dependent: There is no single best way to pick data. A sampling policy that improves performance on a shifted test set (like chemical stratification) often degrades performance on the original in-distribution test set.
- Chemical Divergence is the Best Predictor: The authors found a strong correlation between a model's relative loss and the Kullback–Leibler (KL) divergence between the training and test chemical distributions. If your training set doesn't "look" like your test set chemically, scaling laws won't save you.
- Energy Scales Faster than Forces: Cohesive energy errors drop much more steeply with additional data than force or stress errors. This suggests that "energy-only" scaling studies might give an overly optimistic view of how quickly a model is actually improving for real-world molecular dynamics applications.
Key Results
- Scaling Exponents ($\alpha$): The rate of error decay varied by 1.6–2× between different sampling policies within the same architecture.
- Architecture Sensitivity: While MACE (equivariant) generally outperforms M3GNet (invariant), the gap narrows or even reverses at small data scales (e.g., 10k structures) depending on the sampling policy used.
- Predictive Correlation: Spearman correlations of 0.68–0.89 were observed between model loss and the size-adjusted KL divergence of chemical space, confirming that "relevance" to the test set is a dominant factor in performance.
Limitations and Future Work
- Limited Architectures: The study focused on three specific model configurations. While representative, they don't cover the entire "zoo" of MLIPs.
- Data Range: The training sizes spanned only two orders of magnitude (10k to 1M). Scaling behavior might change at the "extreme" ends (e.g., billions of structures).
- Controlled OOD: The distribution shift in MP-r2SCAN involves both chemical changes and different DFT settings simultaneously, making it hard to isolate exactly which factor causes the performance drop.
- Future Direction: The authors suggest that future scaling reports should always specify the sampling policy and the intended use case, rather than just reporting structure counts. Investigating how magnetic degrees of freedom affect scaling is also a noted area for future research.
Summary updated