Abstract
Universal pre-trained machine learned interatomic potentials (UPMLIPs) such as MACE, CHGNET, and M3GNET offer the potential to perform large-scale materials simulations with the accuracy of ab-initio methods at a fraction of the computational cost. However, their reliability in high-precision thermodynamic applications, such as predicting the stability of disordered alloys, remains to be fully validated. In this work, we evaluate these models against Density Functional Theory (DFT) calculations for 21 binary alloy systems. We find that while UPMLIPs struggle to achieve the "chemical accuracy" required to consistently predict the correct sign of mixing enthalpies out-of-the-box, they serve as excellent structural pre-relaxers and can be significantly improved through targeted fine-tuning.
Introduction: The Promise of Universal Potentials
In alloy theory, the mixing enthalpy ($\Delta H_{mix}$) is a critical parameter that determines whether a binary system will form a solid solution or phase-separate. Traditionally, calculating these values requires DFT, which is computationally expensive and limits the size of the systems that can be studied. The emergence of "universal" potentials—models pre-trained on massive datasets like the Materials Project—promises a shortcut. These models are designed to be used "off-the-shelf" across a wide range of chemistries. This study investigates whether these models can handle the subtle energy differences (often < 50 meV/atom) that govern alloy thermodynamics.
Benchmarking Performance Across Binary Alloys
We benchmarked MACE, CHGNET, and M3GNET against high-fidelity DFT calculations (PBE and LDA functionals) using Special Quasirandom Structures (SQS) to represent disordered states. The evaluation spanned 21 systems at various concentrations.
As shown in the broad benchmarking results, while the models generally capture the overall trend of total energy (left columns), the derived mixing enthalpies (right columns) show significant discrepancies. The Mean Absolute Errors (MAEs) for mixing enthalpy at a 50/50 concentration ranged from 27.7 to 41.5 meV/atom. For context, the difference between two different DFT functionals (LDA and PBE) is typically only ~9 meV/atom, meaning the ML error is often larger than the physical signal being measured.
The Precision Problem: Sign Errors and Stability
A major finding is the "MAE Trap": because mixing enthalpies are often small, an error of 30 meV/atom can lead a model to predict the wrong sign for the energy. This results in a qualitative failure where a model predicts an alloy is stable when it is actually unstable, or vice versa.
In the Cu-Rh system, for example, DFT (PBE) predicts a positive mixing enthalpy, indicating a tendency toward phase separation. However, both CHGNET and M3GNET predict negative values, incorrectly suggesting a stable solid solution. This inconsistency highlights that "universal" does not necessarily mean "ready-to-use" for high-precision thermodynamics without further refinement.
Accelerating Discovery with Hybrid Workflows
Despite the inaccuracies in absolute energy, the models proved highly effective at predicting forces. They can relax atomic structures into the correct geometries almost as accurately as DFT. This leads to a powerful hybrid workflow: using a UPMLIP like MACE to perform the expensive structural relaxation, followed by a single "static" DFT calculation to obtain the final high-precision energy. This approach yields results nearly identical to full DFT relaxation at a significantly reduced computational cost.
Furthermore, we demonstrated that small-scale fine-tuning can correct these errors. For the Al-Zn system, adding just 34 relevant structures to the training set allowed MACE to correctly predict the mixing behavior, provided the training data settings (like pseudopotentials) perfectly matched the original pre-training environment.
Conclusion
While current UPMLIPs lack the out-of-the-box chemical accuracy required for reliable alloy thermodynamics, they are invaluable tools for accelerating traditional workflows. Future foundation models for materials science should incorporate more diverse data, including disordered solid solutions and liquid phases, to bridge the gap between general chemistry and thermodynamic precision. For now, researchers should treat these models as sophisticated starting points rather than total replacements for ab-initio calculations.
Summary updated