Systematic Evaluation of Training Data Generation Strategies for Machine Learning-Based Molecular and Materials Property Prediction

Minh Nghia Nguyen, Beate Paulus, Akhil S. NairarXiv 2026 · arXiv

In the race to discover new molecules and materials, machine learning (ML) has become an indispensable tool. However, training these models often requires massive datasets labeled with high-fidelity simulations (like Density Functional Theory), which are computationally expensive. A new paper by Nguyen et al. systematically explores how we can work smarter, not harder, by using Active Learning (AL) to curate training data more efficiently.

TL;DR

This study benchmarks different active learning strategies across molecular and materials datasets, revealing that uncertainty-based sampling is the most robust way to reduce labeling costs. The authors find that many popular datasets are highly redundant, and by picking the right samples, researchers can achieve near-peak model performance using as little as 13–25% of the available data.

What Problem Does This Paper Address?

Most ML models in chemistry and materials science are trained on datasets curated through "passive" or random sampling. This often leads to a "Goldilocks" problem: some regions of chemical space are over-represented (redundancy), while critical, high-information regions are ignored.

Because generating labels (e.g., calculating a molecule's energy) is the primary bottleneck, we need a way to identify which specific samples will provide the most "knowledge" to the model. This is the core of Active Learning: an iterative process where the model "asks" for the labels of samples it finds most informative. While AL is a known concept, its effectiveness across different model architectures (classical vs. deep learning) and different types of data (molecules vs. crystals) hasn't been systematically compared—until now.

Methods

The researchers compared three primary data acquisition strategies:

  1. Random Sampling (RS): The baseline "passive" approach.
  2. Uncertainty Sampling (US): Using a "query-by-committee" approach. The system picks samples where a committee of models disagrees the most on the predicted value.
  3. Diversity Sampling (DS): A geometric approach (using the TypiClust algorithm) that picks samples to ensure the training set covers the widest possible variety of structures in the feature space.

They tested these strategies using two distinct model classes:

  • Descriptor-based Ensembles: A combination of Random Forest, Kernel Ridge Regression, and XGBoost using traditional chemical descriptors (like fingerprints).
  • Deep Learning: SchNet, a continuous-filter convolutional neural network that learns representations directly from 3D atomic coordinates.

Datasets

The study utilized four datasets to capture different scales and domains:

  • QM9 & QM9small: ~134k small organic molecules (and a 5% subset) for predicting the HOMO–LUMO gap.
  • Matbench Materials Project (MP): ~133k inorganic crystals for predicting formation energy.
  • Pourbaix Stability: A smaller, high-stakes dataset of ~6.8k crystals with expensive HSE06-level labels.
Target property distributions across the four datasets used in the study.

Key Insights and Ideas

  • Massive Redundancy: The most striking "aha moment" is the sheer amount of filler in standard datasets. For the large QM9 and Matbench MP datasets, only 15–25% of the data was needed to reach within 10% of the "full-data" accuracy.
  • Uncertainty Trumps Diversity: In almost every scenario, Uncertainty Sampling (US) outperformed Diversity Sampling (DS). This suggests that model disagreement is a much more potent signal for informativeness than structural variety alone.
  • Model Choice > Sampling Strategy: While AL helps, the choice of model architecture is often more impactful. SchNet (the deep learning model) consistently outperformed the classical ensemble, even when the ensemble was given significantly more data.
  • The "Data-Scarce" Sweet Spot: AL shows its greatest value when data is truly limited and expensive. On the Pourbaix dataset, Uncertainty Sampling saved roughly 1,200 labels (22% of the pool) compared to random sampling to reach the same accuracy threshold.

Key Results

The authors quantified data efficiency using $N_{10\%}$—the number of samples required to reach a performance level within 10% of the model trained on the entire dataset.

  • Efficiency Gains: For the descriptor-based ensemble on QM9, Uncertainty Sampling reached the threshold at 19,000 samples (17% of the pool), whereas Random Sampling required 26,000 (25%).
  • Deep Learning Superiority: SchNet achieved an RMSE of 0.090 eV on QM9, vastly outperforming the ensemble's 0.339 eV. As shown in Figure 6, SchNet is superior at every budget level, though it requires more data to reach its own (much lower) "full-data" error threshold.
Comparison of the best performing strategies for the Ensemble vs. SchNet. SchNet consistently provides a lower error floor across all training sizes.
  • Dataset Specificity: While US was generally best, the "best" strategy for SchNet actually shifted depending on the dataset (US for molecules, DS for crystals), highlighting that there is no "one-size-fits-all" recipe for deep learning.

Limitations and Future Work

The authors noted several constraints:

  • Computational Cost: Due to GPU limits, SchNet models were trained using only a single run per configuration, meaning the statistical variance (error bars) for the deep learning curves could not be rigorously assessed.
  • Limited Strategy Suite: The study focused on three core strategies. Future work could explore hybrid approaches that combine uncertainty and diversity.
  • Benchmarking Platform: The authors aim to develop a broader benchmarking platform to help the community quantify redundancy in new target properties and select the best AL strategies for their specific budgets.

For researchers in the field, the takeaway is clear: if you have a limited labeling budget, start with uncertainty-based sampling and, if structural data is available, prioritize deep learning architectures like SchNet over classical ensembles.

Summary updated

Abstract

Machine-learning (ML) models for molecular and materials property prediction are often trained on large datasets curated through arbitrary or generic sampling strategies. Such approaches can lead to the over- or under-representation of regions of chemical and materials space that are most relevant to the prediction task. An important challenge is therefore how to efficiently curate training data to maximize predictive performance while minimising the number of labelled examples required. Here, we systematically compare training data acquisition strategies for molecular and materials property prediction, ranging from random sampling to various active learning approaches. We evaluate their performance across different model classes, representations, datasets, and target properties. Our results show that the effectiveness of data acquisition strategies depend strongly on the interplay between the model, dataset, and prediction task, although uncertainty-based sampling generally outperforms alternative strategies. Based on these findings, we derive guidelines for selecting data acquisition strategies under a given labelling budget.
BibTeX
@misc{nguyen2026systematic,
  author = {Minh Nghia Nguyen and Beate Paulus and Akhil S. Nair},
  title = {{Systematic Evaluation of Training Data Generation Strategies for Machine Learning-Based Molecular and Materials Property Prediction}},
  year = {2026},
  eprint = {2611.16792},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16792},
}
Figure