Integration of generative machine learning with the heuristic crystal structure prediction code FUSE

Christopher M. Collins, Hasan M. Sayeed, George R. Darling, John B. Claridge, Taylor D. Sparks, Matthew J. Rosseinsky

Abstract

The prediction of new compounds via crystal structure prediction (CSP) may transform how the materials chemistry community discovers new compounds. In the prediction of inorganic crystal structures, there are three distinct classes of prediction: performing CSP via heuristic algorithms using a range of established codes, an emerging community using generative machine learning (ML) models to predict crystal structures directly, and the use of mathematical optimization to solve crystal structures exactly. In this work, we demonstrate the combination of heuristic and generative machine learning, using a generative ML model to produce the starting population of crystal structures for a heuristic algorithm. We demonstrate the method on eight known compounds with reported crystal structures and three hypothetical compounds. We show that the integration of machine learning structure generation with heuristic structure prediction results in both faster compute times per structure and lower energies. This work provides a set of eleven compounds with varying chemistry and complexity that can be used as a benchmark for new CSP methods as they emerge.

Introduction

The discovery of novel materials is a fundamental challenge in chemistry and materials science, often revolving around the determination of stable atomic arrangements for a given chemical composition. Crystal Structure Prediction (CSP) has emerged as a powerful tool to address this, aiming to identify the global energy minimum of a potential energy surface. Traditionally, CSP has relied on heuristic algorithms, such as genetic algorithms or basin hopping, which explore the structural landscape iteratively. While effective, these methods are computationally demanding, often starting from random, unphysical configurations that require extensive relaxation.

Recently, generative machine learning (ML) models have offered a faster alternative, capable of generating plausible structural candidates almost instantaneously. However, these models often lack the precision required to reach true physical equilibrium or struggle with chemical spaces outside their training data. This paper introduces a hybrid approach, integrating generative ML with the heuristic search engine FUSE (Flexible Unit Structure Engine), to leverage the speed of ML and the robustness of heuristic refinement.

Methodology: The FUSE2 Framework

The authors present FUSE2, an advanced iteration of the FUSE code that incorporates a generative Graph Network (GN) model into its workflow. The process begins with ML Generation, where a GN model trained on the Matbench dataset generates a diverse pool of candidate structures for a target composition.

To ensure computational efficiency, these candidates undergo Ranking and Filtering. This step utilizes Statistical Proxy Potentials (SPP) or ChgNET, a pretrained universal neural network potential, to identify the most promising structures and discard unphysical outputs. The top-ranked structures then serve as the initial population for Heuristic Refinement within FUSE2. Unlike traditional methods that start from random coordinates, FUSE2 decomposes the ML-generated structures into sub-modules and employs a basin-hopping routine to navigate the local energy landscape. Finally, the refined structures are subjected to high-accuracy Local Optimization using Density Functional Theory (DFT) calculations via VASP to determine the precise ground state.

Benchmarking and Datasets

The ML components of the framework were trained using data from the Open Quantum Materials Database (OQMD) and the Matbench structure-energy dataset. To rigorously evaluate the performance of FUSE2, the authors established a new benchmark suite consisting of 11 compounds of varying chemical complexity, with primitive cells containing up to 50 atoms.

The benchmark includes:

  • Eight Known Compounds: Such as $Ca_3Ti_2O_7$, $Cu_7S_4$, and the highly complex sodium iron borosilicate ($Si_6Al_6B_3Fe_3NaO_{30}F$).
  • Three Hypothetical Compounds: Including $Li_2Sn_2S_3Cl_4$, which represent materials previously identified as potential candidates for synthesis.

Key Results and Discussion

The integration of ML "seeds" into the heuristic search yielded significant improvements in both efficiency and accuracy. For six of the eight known compounds, the hybrid ML-FUSE2 approach outperformed traditional random-start methods in terms of speed. Notably, for $WCl_2$, the researchers observed an 8.3x speedup. On average, the hybrid method was approximately 2.2 times faster across the entire benchmark set.

Beyond speed, the hybrid approach consistently identified more stable configurations. In 58 out of 66 experimental runs, the heuristic search successfully lowered the energy of the initial ML-generated structure, highlighting the necessity of the refinement step. For the complex $Si_6Al_6B_3Fe_3NaO_{30}F$, the hybrid method discovered a structure with an energy 338 meV per atom lower than the baseline random-start approach.

However, the study also identified limitations. In the case of $YWB_4$, the ML-ranking process actually hindered the search. The authors suggest that the generic proxy potentials used for ranking may not have accurately represented the specific chemistry of $YWB_4$, leading the algorithm toward local minima far from the true DFT ground state. This underscores the need for more specialized or adaptive ranking potentials.

Conclusion

This work demonstrates that hybridizing generative machine learning with heuristic search algorithms significantly enhances the capability of crystal structure prediction. By providing a high-quality "warm start," ML reduces the computational burden of DFT relaxations and allows heuristic codes to explore more relevant regions of the energy landscape. The curated benchmark of 11 compounds serves as a valuable resource for the community to test and compare emerging CSP methodologies. Future work will focus on developing chemistry-specific ranking potentials and incorporating reinforcement learning to further optimize structural search strategies.

Summary updated

Abstract

The prediction of new compounds <i>via</i> crystal structure prediction may transform how the materials chemistry community discovers new compounds. In the prediction of inorganic crystal structures there are three distinct classes of prediction: performing crystal structure prediction <i>via</i> heuristic algorithms, using a range of established crystal structure prediction codes, an emerging community using generative machine learning models to predict crystal structures directly and the use of mathematical optimisation to solve crystal structures exactly. In this work, we demonstrate the combination of heuristic and generative machine learning, the use of a generative machine learning model to produce the starting population of crystal structures for a heuristic algorithm and discuss the benefits, demonstrating the method on eight known compounds with reported crystal structures and three hypothetical compounds. We show that the integration of machine learning structure generation with heuristic structure prediction results in both faster compute times per structure and lower energies. This work provides to the community a set of eleven compounds with varying chemistry and complexity that can be used as a benchmark for new crystal structure prediction methods as they emerge.
BibTeX
@inproceedings{collins2024integration,
  author = {Christopher M. Collins and Hasan M. Sayeed and George R. Darling and John B. Claridge and Taylor D. Sparks and Matthew J. Rosseinsky},
  title = {{Integration of generative machine learning with the heuristic crystal structure prediction code FUSE}},
  booktitle = {Faraday},
  year = {2024},
  doi = {10.1039/d4fd00094c},
}
Figure