CrysToGraph: A Comprehensive Predictive Model for Crystal Materials Properties and the Benchmark

Hongyi Wang, Ji Ung Sun, Jinzhe Liang, Li Zhai, Zitian Tang, Zijian Li, Wei Zhai, Xusheng Wang, Weihao Gao, Shaohua Gong

TL;DR

CrysToGraph is a hybrid geometric graph network that combines local message-passing with global transformer layers to capture both short-range atomic bonds and long-range structural orders in crystals. Alongside the model, the authors introduce UnconvBench, a new benchmark specifically designed to evaluate how well models handle "unconventional" materials like 2D layers, porous MOFs, and defected structures.

What Problem Does This Paper Address?

Traditional Graph Neural Networks (GNNs) have become the workhorse for predicting material properties because they naturally represent atoms as nodes and bonds as edges. However, GNNs are inherently "nearsighted." Because they rely on local message-passing, they struggle to capture the long-range periodic orders and symmetries that define a crystal's macroscopic behavior.

This limitation is particularly acute in "unconventional" crystalline systems:

  1. 2D Materials: Where ultra-thin layers create unique electronic environments.
  2. Metal-Organic Frameworks (MOFs): Which have massive unit cells and porous structures.
  3. Defected Crystals: Where a single missing or misplaced atom can disrupt long-range stability.

Existing benchmarks often focus on simple, highly ordered bulk crystals, leaving a gap in our ability to evaluate models on these more complex, real-world materials.

Methods

CrysToGraph employs a dual-scale architecture to bridge the gap between local chemistry and global structure:

  • eTGC (edge-engaged Transformer Graph Convolution): This component handles short-range interactions. Unlike standard GNNs, it uses a line graph construction to explicitly model three-body interactions (angles between bonds). It updates node and edge features simultaneously using a shared attention mechanism.
  • GwT (Graph-wise Transformer): To capture long-range dependencies, the model treats all nodes in the crystal as a sequence for a global transformer. This allows every atom to "attend" to every other atom in the unit cell, regardless of distance.
  • Hybrid Positional Encodings: Since transformers are permutation-invariant, the authors use a combination of Laplacian and Random Walk positional encodings. These provide the global transformer with the structural and connectivity context it needs to understand the crystal's "map."
  • Architecture Flow: The model typically stacks several eTGC blocks followed by a GwT layer, effectively performing local feature extraction before global reasoning.

Datasets

The paper evaluates CrysToGraph on two primary fronts:

  1. UnconvBench (New): A curated collection of 11 datasets focusing on unconventional systems. It includes tasks for 2D crystal exfoliation energy, MOF gas adsorption ($CO_2$), and formation energies for defected crystals. It also includes "Bulk S/M/L" datasets to test performance as unit cell size increases.
  2. MatBench: The industry-standard benchmark for traditional bulk materials (e.g., formation energy, band gap, shear modulus).

Key Insights and Ideas

  • The "Nearsightedness" Cure: The authors demonstrate that while GNNs are sufficient for small, simple crystals, the global attention of the GwT becomes exponentially more important as the unit cell size increases (e.g., in MOFs).
  • Angles Matter: By using line graphs in the eTGC blocks, the model captures geometric information (bond angles) that simple edge-distance models miss, which is critical for structural stability.
  • Stacking vs. Parallel: Through ablation studies, the researchers found that stacking (eTGC then GwT) is more effective and computationally efficient than running them in parallel. The local features act as a "refined" input for the global transformer.
  • Line Graphs for Defects: Interestingly, the full model (with line graphs) performed significantly better on defected crystals than on pure bulk crystals, suggesting that explicit structural modeling is key to identifying the subtle stresses caused by lattice defects.

Key Results

  • State-of-the-Art (SOTA): CrysToGraph achieved SOTA results on 10 out of 15 total datasets tested.
  • UnconvBench Performance: It outperformed established models like ALIGNN and CGCNN across nearly all unconventional tasks, showing a particularly massive lead in the qmof (MOF formation energy) and co2_adsp (adsorption) tasks.
  • MatBench Performance: Even on traditional materials, it remained highly competitive, securing SOTA or second-place results in 5 out of 8 tasks, proving it is a robust general-purpose model.
  • Correlation: In the $CO_2$ adsorption task, the model achieved an $r^2$ score of 0.8946, demonstrating high reliability for complex chemical property prediction.

Limitations and Future Work

  • Computational Scaling: While the authors optimized the architecture, fully connected transformers (GwT) naturally face quadratic scaling issues as the number of atoms in a unit cell grows extremely large.
  • Dynamic Simulations: The current model is a static property predictor. The authors suggest that CrysToGraph could be extended into a machine-learning force field for molecular dynamics (MD) simulations, which would allow for studying materials at finite temperatures.
  • Hyperparameter Sensitivity: The "ideal" depth of the eTGC blocks (typically 3-7) varies by dataset, suggesting that the model requires some tuning depending on whether the target property is more local or global in nature.

Summary updated

Abstract

The ionic bonding across the lattice and ordered microscopic structures endow crystals with unique symmetry and determine their macroscopic properties. Unconventional crystals, in particular, exhibit non-traditional lattice structures or possess exotic physical properties, making them intriguing subjects for investigation. Therefore, to accurately predict the physical and chemical properties of crystals, it is crucial to consider long-range orders. While GNN excels at capturing the local environment of atoms in crystals, they often face challenges in effectively capturing longer-ranged interactions due to their limited depth. In this paper, we propose CrysToGraph ($\textbf{Crys}$tals with $\textbf{T}$ransformers $\textbf{o}$n $\textbf{Graph}$s), a novel transformer-based geometric graph network designed specifically for unconventional crystalline systems, and UnconvBench, a comprehensive benchmark to evaluate models' predictive performance on unconventional crystal materials such as defected crystals, low-dimension crystals and MOF. CrysToGraph effectively captures short-range interactions with transformer-based graph convolution blocks as well as long-range interactions with graph-wise transformer blocks. CrysToGraph proofs its effectiveness in modelling unconventional crystal materials in multiple tasks, and moreover, it outperforms most existing methods, achieving new state-of-the-art results on the benchmarks of both unconventional crystals and traditional crystals.
BibTeX
@inproceedings{wang2025crystograph,
  author = {Hongyi Wang and Ji Ung Sun and Jinzhe Liang and Li Zhai and Zitian Tang and Zijian Li and Wei Zhai and Xusheng Wang and Weihao Gao and Shaohua Gong},
  title = {{CrysToGraph: A Comprehensive Predictive Model for Crystal Materials Properties and the Benchmark}},
  booktitle = {Battery},
  year = {2025},
  doi = {10.1002/bte2.70004},
}
Figure