In computational chemistry and materials science, simulating how atoms interact has traditionally required a difficult trade-off. To achieve high accuracy, researchers relied on quantum mechanical simulations like Density Functional Theory (DFT). However, DFT is computationally expensive, scaling poorly to large systems or long simulation times.
Machine Learning Interatomic Potentials (MLIPs) emerged to solve this by fitting neural networks to DFT data, acting as fast surrogates. Yet, early MLIPs suffered from a major limitation: they were strictly "bespoke" models. If you wanted to simulate a new alloy, a specific crystal defect, or a unique chemical reaction, you had to generate thousands of new, expensive DFT data points to train a model from scratch for that exact system.
Recently, the field has undergone a paradigm shift. Instead of training isolated models for specific chemical systems, researchers are building atomistic foundation models. By pre-training large neural networks on massive, diverse datasets across the periodic table, these models learn universal chemical priors. Researchers can then transfer this knowledge to data-scarce downstream tasks, achieving high-accuracy simulations with a fraction of the data previously required.
Why Pre-training Works: Capturing Universal Chemical Priors
The core intuition behind transfer learning in atomistic modeling is that atoms obey the same fundamental physical laws regardless of the specific material they comprise. An electron configuration trend or a local bonding environment in one crystal structure shares deep physical commonalities with those in another. Pre-training allows a neural network to learn these underlying rules of the periodic table.
Large Atomic Models (LAMs) exploit this shared physics by training across vast chemical spaces. For example, DPA-2, a foundation model developed by Duo Zhang et al., utilizes multi-task pre-training across 73 chemical elements. By learning a universal representation of atomic environments, the model internalizes periodic trends, allowing it to generalize across diverse simulation tasks. This builds upon earlier attention-based architectures like DPA-1, where Duo Zhang et al. demonstrated that large-scale pre-training dramatically enhances sample efficiency, reducing the volume of target data needed by up to an order of magnitude.
To maximize the physical fidelity of these models, researchers are also rethinking what information the model ingests during pre-training. While standard MLIPs train only on atomic coordinates, energies, and forces, Manasa Kaniselvan et al. introduced HELM, a framework that pre-trains directly on the electronic Hamiltonian matrix. By learning from the underlying electronic structure rather than just the resultant forces, the model acquires a richer physical prior, doubling the accuracy of molecular energy predictions in low-data regimes.
This shift toward massive pre-training requires architectures designed for scale. Eric Qu argues that instead of hard-coding rigid physical symmetries into neural networks—which can bottleneck computational throughput—prioritizing model scalability and optimized attention mechanisms allows foundation models to ingest diverse chemical datasets far more effectively, ultimately yielding superior transferability.
Downstream Specialization via Fine-Tuning
Once an atomistic foundation model has acquired a broad understanding of chemistry, it can be adapted to highly specialized tasks through fine-tuning. During this phase, the model's weights are slightly adjusted using a small, targeted dataset from the system of interest.
Because the model already understands general atomic interactions, it does not need to relearn basic physics; it only needs to learn the local nuances of the target system. This makes the process incredibly data-efficient. For instance, Lingyu Kong et al. developed MatterTune, a platform standardizing the fine-tuning of foundation models like MACE and EquformerV2. The authors demonstrated that fine-tuning an atomistic foundation model on as few as 30 samples can successfully reproduce complex experimental properties, such as radial distribution functions.
[ Massive, Diverse Chemical Data ] ---> Pre-trained Foundation Model (Learns Periodic Trends)
|
v + (Very Small Target Dataset, e.g., ~30 samples)
Fine-tuned Specialized Model (Accurate Local Predictions)
This data efficiency unlocks simulations in complex chemical spaces where generating large datasets is computationally impossible:
- Perovskite Oxides: Jing Wu et al. developed UniPero, a universal potential for perovskites. By leveraging a pre-trained backbone, they successfully simulated complex phase transitions, surface defects, and grain boundaries across 14 different metal elements with minimal additional data.
- Complex Alloys: Simulating multi-element systems is notoriously difficult due to the combinatorial explosion of atomic configurations. Xiangfu Niu et al. fine-tuned foundation models on domain-specific alloy data, enabling high-fidelity simulations of disorder-to-order transitions and atomic migration barriers in PtCo-based ternary alloys.
Correcting the Blind Spots of Universal Models
While universal pre-trained models provide an exceptional baseline, they can occasionally struggle with highly subtle thermodynamic properties or out-of-distribution structures. Fine-tuning acts as the necessary bridge to correct these blind spots.
As evaluated by Luis Casillas‐Trujillo et al., zero-shot universal potentials (such as MACE or CHGNET) sometimes struggle to predict the exact sign or magnitude of mixing enthalpies in disordered binary alloys. However, the authors showed that targeted retraining with a small amount of system-specific data corrects these errors, combining the broad structural stability of the foundation model with the high precision required for thermodynamic mixing properties.
A similar challenge occurs when mapping phase diagrams or predicting crystal structures, where models must evaluate "unrelaxed," highly unstable intermediate atomic arrangements. Filip Ekström Kelvinius et al. addressed this by performing transfer learning between chemically analogous systems—such as elements within the same periodic group. By exploiting these inherent physical similarities, they reduced the computational cost of mapping new phase diagrams by over 70%.
Similarly, in the context of crystal structure prediction, Chang Liu et al. pre-trained a Crystal Graph Convolutional Neural Network on global datasets and fine-tuned it on specific compositions, enabling rapid and accurate energy predictions for unrelaxed structures. This strategy of refining universal graph neural networks with high-fidelity local calculations was also leveraged by Guillermo Vazquez et al. to accurately decipher complex chemical ordering in high-entropy materials.
Broadening the Horizon: Generative Design and Bio-Endpoints
The success of transfer learning has led to open-source ecosystems like MatGL, developed by Bowen Deng et al., which integrates pre-trained invariant and equivariant graph neural networks to lower the barrier for downstream property prediction. Today, this paradigm extends far beyond static interatomic potentials, transforming generative materials design and biochemistry.
In generative modeling, Claudio Zeni et al. developed MatterGen, a foundational diffusion model for inorganic materials. By pre-training on large-scale unlabeled structural data, the model learns the syntax of stable crystal lattices. When fine-tuned on minimal labeled examples, MatterGen can be "steered" to generate entirely novel materials optimized for specific mechanical, magnetic, or electronic properties.
[ Unlabeled Crystal Structures ] ---> MatterGen Pre-training (Learns Stable Crystal Syntax)
|
v + (Minimal Labeled Property Data)
Fine-tuned Generator (Designs Target Hardness/Magnetism)
The same data-efficient adaptation principles apply to drug discovery and biochemical endpoints. Wei Chen et al. demonstrated that pre-trained molecular encoders can learn generalizable molecular "fingerprints." When adapted to new, unseen viral protein targets, these models achieve high predictive accuracy even when the available target dataset is extremely limited.
Similarly, Shanzhuo Zhang et al. developed HelixADMET, pre-training a chemical backbone on 20 million unlabeled molecules. This foundational chemical knowledge transfers seamlessly to niche biological endpoints, outperforming models trained from scratch on small, specialized datasets. Finally, when combined with active learning, as shown by Ambuj Tewari et al., this transfer learning paradigm allows researchers to optimize challenging chemical reactions and boost synthetic yields in just a few iterations, proving that a strong foundational prior is the key to navigating data-scarce chemical landscapes.
Key Foundational Work
Universal interatomic potential for perovskite oxides (2023) — contributed 2 concept elements to this direction.
Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials (2023) — contributed 1 concept elements to this direction.
HelixADMET: a robust and endpoint extensible ADMET system incorporating self-supervised knowledge transfer (2022) — contributed 1 concept elements to this direction.