Vector Field Oriented Diffusion Model for Crystal Material Generation

Astrid Klipfel, Yaël Frégier, Adlane Sayede, Zied BouraouiAAAI 2024 · DOI

TL;DR

This paper introduces GemsDiff, a novel diffusion model that generates crystal structures by considering both atomic positions and lattice geometry simultaneously. By leveraging an equivariant graph neural network and a diffusion process on a torus, GemsDiff achieves state-of-the-art results in generating realistic crystal lattices with specific chemical compositions, addressing a key limitation in current material discovery methods.

What Problem Does This Paper Address?

Discovering new crystal materials with specific properties is crucial for technological advancements, particularly in areas like renewable energy and semiconductors. However, existing machine learning models for crystal generation often struggle with accurately representing and generating the complex periodic nature of crystal lattices. Many models either focus solely on atomic positions or chemical composition, or they impose constraints that limit their ability to explore diverse crystal structures and compositions. This makes it challenging to perform precise sampling for specific chemical compositions, which is essential for tasks like constructing convex hulls of material phases. The paper aims to bridge this gap by proposing a generative model that can handle the full geometry of crystal structures, including both atomic arrangements and the unit cell, while allowing for precise compositional sampling.

Methods

The core of this work is a probabilistic diffusion model called GemsDiff. It tackles crystal generation in two main steps:

  1. Diffusion on a Torus: To handle the inherent periodicity of crystal lattices, the model first defines a diffusion process on a torus. This allows for the diffusion of atomic positions within the periodic boundaries of the unit cell.
  2. Equivariant Graph Neural Network (GNN): A key innovation is the use of a geometrically equivariant GNN. This GNN takes the crystal's geometry (lattice and atomic positions) and chemical composition as input and learns to reverse the diffusion process. Crucially, this GNN is designed to be equivariant to transformations of the crystal lattice, ensuring that the model's predictions are consistent with the underlying symmetries of crystal structures. The model also incorporates a diffusion process on the lattice itself, which is a significant departure from previous methods that only diffused atomic positions.

The model's loss function is designed to capture both lattice reconstruction and atomic position refinement, ensuring that the generated structures are both geometrically sound and physically plausible.

Datasets

The experiments were conducted on a subset of the Materials Project dataset. This dataset contains crystalline materials studied with ab-initio calculations. The authors filtered the dataset to include only stable materials with fewer than 20 atoms, aligning with previous work (Xie et al., 2021) for comparability. They also used the same training, validation, and test splits as (Xie et al., 2021).

Key Insights and Ideas

  • Joint Diffusion of Atoms and Lattice: The most significant insight is the successful application of diffusion models to both atomic positions and the unit cell geometry simultaneously. This holistic approach is crucial for generating realistic crystal structures, as the unit cell dictates periodicity and density, which directly impact material properties.
  • Equivariance for Geometric Consistency: The use of an equivariant GNN is a powerful technique for ensuring that the generative model respects the geometric symmetries inherent in crystal structures. This leads to more stable and physically meaningful predictions.
  • Torus Diffusion for Periodicity: Adapting diffusion to a toroidal space is an elegant solution for handling the periodic boundary conditions of crystals, a challenge that has limited previous generative models.
  • Emergent Structure Generation: The paper highlights that GemsDiff can generate complex atomic arrangements (e.g., alignments, rings) and adhere to specific lattice systems (e.g., hexagonal) without explicit rules or constraints, suggesting that these desirable properties emerge naturally from the diffusion process and the equivariant architecture.
  • New Evaluation Metric (FAD): The introduction of Frechet ALIGNN Distance (FAD) as a more comprehensive metric for crystal evaluation, inspired by computer vision's FID, is a valuable contribution. It moves beyond simple validity, density, and energy comparisons to assess the overall distribution of generated crystal properties.

Key Results

  • State-of-the-Art Performance: GemsDiff outperforms several state-of-the-art baselines (CDVAE, FTCP, PGCGM, G-SchNet, P-G-SchNet) on key metrics.
  • Improved Density and Energy: The model achieves significantly better Earth Mover Distance (EMD) for density (0.3331 ± 0.0327) and formation energy (0.1984 ± 0.0086) compared to CDVAE and other baselines, indicating more accurate generation of physically realistic structures.
  • High Validity: The model achieves a high validity score (99.05 ± 0.20%), indicating that most generated structures are physically plausible.
  • Competitive FAD: GemsDiff achieves the best Frechet ALIGNN Distance (FAD) score (1.509 ± 0.115), suggesting that its generated crystal structures are closer to equilibrium positions (i.e., more stable) than those produced by other models.
  • Efficient Sampling: The sampling process for the test set (9046 crystals) takes approximately 45 minutes, which is considered normal for a diffusion model.

Limitations and Future Work

  • Prioritization of Specialized Tasks: The authors acknowledge that their model is less suited for generating compositions without specific constraints compared to models designed for that purpose. However, they emphasize its strength in generating structures for specific chemical compositions, which is valuable for tasks like convex hull construction.
  • Future Work on Space Groups: The paper suggests that decomposing the lattice into a Lie algebra and using invariant vector fields could be extended to construct structures belonging to particular space groups, guaranteeing lattice compatibility with those groups.
  • FAD Consistency: While FAD is a promising metric, the authors note the importance of using pre-trained versions of models like ALIGNN for consistent FAD calculations, as retraining can affect feature consistency.
  • Ablation Study: An ablation test is proposed in the appendix to further justify the architecture's components.

Summary updated

BibTeX
@inproceedings{klipfel2024vector,
  author = {Astrid Klipfel and Yaël Frégier and Adlane Sayede and Zied Bouraoui},
  title = {{Vector Field Oriented Diffusion Model for Crystal Material Generation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i20.30224},
}
Figure