Neural networks trained on synthetically generated crystals can extract structural information from ICSD powder X-ray diffractograms

Henrik Schopmans, Patrick Reiser, Pascal Friederich

TL;DR

Researchers have developed a method to train deep neural networks for crystal space group classification using an infinite stream of "chemically incorrect" but "symmetrically valid" synthetic crystals. By bypassing the limited and biased data in existing databases, their ResNet-101 model achieved 79.9% accuracy on unseen structure types, vastly outperforming models trained on real database entries.

What Problem Does This Paper Address?

Identifying the space group of a material from its powder X-ray diffraction (XRD) pattern is a fundamental step in materials science, but it usually requires expert manual analysis. While machine learning (ML) offers a path to automation, it faces a "small data" problem.

The standard source for training, the Inorganic Crystal Structure Database (ICSD), contains roughly 270,000 entries. While this sounds large, the data is highly imbalanced—some space groups have thousands of examples, while others have fewer than 50 (see Figure 1). Furthermore, many entries are just slight variations of the same common structure types (like NaCl). If a model is trained and tested on a random split of this data, it often just "memorizes" specific structure types rather than learning the underlying physics of diffraction. This leads to poor generalization when the model encounters a truly new material.

Figure 1: Distribution (logarithmic scale in blue, linear scale in red) of space groups in the ICSD. Space groups are sorted by count (see Figure S13 in the SI for the distribution without sorting by count). The population of the space groups varies by multiple orders of magnitude, showing that the ICSD is a highly imbalanced dataset regarding space groups. The space groups excluded due to insufficient statistics are visualized with black stripes. The histogram displays the distribution of the full ICSD, while the exclusion of space groups that do not contain enough samples is based on the statistics dataset (which does not include the test dataset, see Section 2.4) that we used to guide the random crystal generation. Therefore, the excluded space groups are not exactly the first 85 counted from the left.

Methods

The authors propose a radical shift: stop training on real crystals and start training on synthetic ones.

  1. Synthetic Generation: Instead of using ICSD entries as training samples, they use the ICSD only to extract broad statistics (e.g., which elements are common, typical lattice parameter ranges). They then generate synthetic crystals by randomly placing atoms on the Wyckoff positions of a given space group.
  2. "Symmetrically Correct, Chemically Incorrect": Crucially, these synthetic crystals don't have to be stable or even physically possible (atoms can overlap). The goal is to teach the network the mathematical relationship between symmetry operations and diffraction peaks, not chemistry.
  3. Online Training at Scale: Using the Ray distributed computing framework, the team generates and simulates diffractograms on-the-fly during training. This creates an "infinite" dataset where the model never sees the same sample twice, effectively eliminating overfitting. They were able to generate up to millions of unique diffractograms per hour.
  4. Deep Learning Architecture: Because they had access to infinite data, they could move beyond simple CNNs to much deeper ResNet models (ResNet-10, 50, and 101) adapted for 1D XRD data.
Figure 2: a) Flowchart of how the generation algorithm produces synthetic crystals. Atoms are independently placed on the Wyckoff positions and random coordinates are drawn. b) Overview of the distributed computing system implemented using the Python library Ray 25. Two compute nodes (that generate and simulate diffractograms) are connected to the Ray head node using a Ray queue object.

Datasets

  • ICSD (Inorganic Crystal Structure Database): Used to extract statistics for the synthetic generator and as the primary test set to evaluate generalization to real-world structures.
  • RRUFF Database: A database of mineral samples used to test the model's performance on raw, experimental data containing noise and impurities.
  • Synthetic Dataset: An infinite, on-the-fly generated stream of simulated diffractograms used exclusively for training.

Key Insights and Ideas

  • Generalization through Randomness: By training on a vast variety of "unstable" synthetic structures, the model learns more general features of symmetry. It becomes robust enough to classify stable structures it has never seen before.
  • The "Structure Type" Trap: The authors demonstrate that standard "random" train-test splits in crystallography are misleading. They advocate for a strict structure type-based split, which ensures the test set contains entirely different crystal architectures than the training set.
  • Symmetry is the Key: The model doesn't need to know if a crystal can exist in nature to learn how its symmetry influences its XRD pattern. Figure 3 shows that while synthetic crystals look "messy" compared to ICSD entries, their symmetry is perfectly preserved.
Figure 3: Comparison of real ICSD crystals (a) and their synthetic counterparts (b).

Key Results

  • Superior Accuracy: The ResNet-101 model trained on synthetic data achieved 79.9% accuracy on unseen ICSD structure types. For comparison, a state-of-the-art model trained directly on ICSD data achieved only 56.1% under the same strict testing conditions.
  • Top-k Performance: The model’s Top-5 accuracy reached 96%, meaning the correct space group is almost always among the model's first few guesses. This is incredibly useful for researchers as a "shortlist" for further refinement.
  • Experimental Validation: When applied to the RRUFF mineral database (experimental data), the model achieved a Top-10 accuracy of over 60%. While lower than the simulated results, it proves the methodology can bridge the gap to real-world laboratory measurements.

Limitations and Future Work

  • The Coordinate Gap: There is still a ~12% gap between training accuracy (on synthetic data) and test accuracy (on ICSD). The authors found this is largely because real crystals have more "ordered" atom positions than their purely random synthetic ones. Future work will focus on generating more "ordered" synthetic structures.
  • Experimental Imperfections: Real-world XRD patterns include background noise, peak broadening, and impurity phases. While the authors began addressing this by adding noise to their simulations, more sophisticated generative models (like GANs) might be needed to perfectly mimic experimental conditions.
  • Beyond Space Groups: The authors suggest this "infinite synthetic data" approach could be extended to other tasks, such as predicting lattice parameters or crystallite size.

Summary updated

Abstract

Machine learning techniques have successfully been used to extract structural information such as the crystal space group from powder X-ray diffractograms. However, training directly on simulated diffractograms from databases such as the ICSD is challenging due to its limited size, class-inhomogeneity, and bias toward certain structure types. We propose an alternative approach of generating synthetic crystals with random coordinates by using the symmetry operations of each space group. Based on this approach, we demonstrate online training of deep ResNet-like models on up to a few million unique on-the-fly generated synthetic diffractograms per hour. For our chosen task of space group classification, we achieved a test accuracy of 79.9% on unseen ICSD structure types from most space groups. This surpasses the 56.1% accuracy of the current state-of-the-art approach of training on ICSD crystals directly. Our results demonstrate that synthetically generated crystals can be used to extract structural information from ICSD powder diffractograms, which makes it possible to apply very large state-of-the-art machine learning models in the area of powder X-ray diffraction. We further show first steps toward applying our methodology to experimental data, where automated XRD data analysis is crucial, especially in high-throughput settings. While we focused on the prediction of the space group, our approach has the potential to be extended to related tasks in the future.
BibTeX
@inproceedings{schopmans2023neural,
  author = {Henrik Schopmans and Patrick Reiser and Pascal Friederich},
  title = {{Neural networks trained on synthetically generated crystals can extract structural information from ICSD powder X-ray diffractograms}},
  booktitle = {Digital Discovery},
  year = {2023},
  url = {https://openreview.net/forum?id=pkKr1kFUB9},
}
Figure