Report on scipost_202203_00027v1

Jiewei Ding, Ho-Kin Tang, Wing Chi YuOpenalex Unknown 2022 · DOI · OpenAlex

Abstract

We found that Bidirectional LSTM and Transformer can classify different phases of condensed matter models and determine the phase transition points by learning features in the Monte Carlo raw data before equilibrium. Our method can significantly reduce the time and computational resources required for probing phase transitions as compared to the conventional Monte Carlo simulation. We also provide evidence that the method is robust and the performance of the deep learning model is insensitive to the type of input data (we tested spin configurations of classical models and green functions of a quantum model), and it also performs well in detecting Kosterlitz-Thouless phase transitions.

Introduction

In condensed matter physics, Monte Carlo (MC) simulations are essential for exploring phase transitions. However, they are computationally expensive because systems must reach thermal equilibrium before valid measurements can be taken. Near phase transitions, "critical slowing down" further increases the time required for convergence. This study investigates whether sequence-based deep learning models can identify phase transitions using only raw, pre-equilibrium data from the earliest stages of a simulation.

Methodology

The researchers treat early MC steps as a time-series problem rather than a spatial one. Instead of feeding a full lattice snapshot to a Convolutional Neural Network (CNN), they track the states of a small number of randomly selected spatial sites over the first 10 to 40 MC steps.

Deep learning architecture overview including Bi-LSTM and Transformer components.

The architecture uses Bidirectional LSTMs (Bi-LSTM) or Transformers to process this sequential data. The models are trained as binary classifiers on data from temperatures far from the critical point ($T_c$). During inference, the transition point is identified where the model's output probability is approximately 50%, indicating maximal uncertainty.

Datasets and Models

The approach was tested on several physical models:

  • 2D Ising Model: Square, Honeycomb, and Triangle lattices.
  • 2D XY Model: Known for the topological Kosterlitz-Thouless (KT) transition.
  • Quantum Hubbard Model: A strongly correlated system where Green's function matrices were used as input.

Key Insights

  • Temporal Features: Sequence models significantly outperform spatial models like CNNs on pre-equilibrium data. By focusing on the evolution of a few sites, the models capture transition dynamics that are obscured by noise in a full spatial grid.
  • Topological Robustness: The models successfully detected the KT transition in the XY model using only local time-evolution data, which is surprising given that topological transitions are typically associated with global spatial features.
  • Order Independence: Shuffling the MC steps did not degrade performance, suggesting the models learn the distribution of state changes inherent in the Markov chain process rather than a strict temporal sequence.

Results

The Bi-LSTM and Transformer models accurately predicted the transition temperature for the classical Ising model ($T_c \approx 2.269$) using only the first 10 to 20 MC steps, a massive reduction from the ~1000 steps typically required for equilibrium. In the Quantum Hubbard model, the transition was identified using only 0.01% of the steps normally needed. While CNNs required more data and had higher error margins, fully connected networks failed to classify the pre-equilibrium data entirely.

Conclusion and Future Work

While highly effective, the method showed some noise in the Quantum Hubbard model due to severe quantum fluctuations in pre-equilibrium Green's functions. Future research could apply this sequence-based approach to real time-ordered simulation data, such as molecular dynamics, to study disordered systems or glass transitions.

Summary updated

Abstract

We found that Bidirectional LSTM and Transformer can classify different phases of condensed matter models and determine the phase transition points by learning features in the Monte Carlo raw data before equilibrium. Our method can significantly reduce the time and computational resources required for probing phase transitions as compared to the conventional Monte Carlo simulation. We also provide evidence that the method is robust and the performance of the deep learning model is insensitive to the type of input data (we tested spin configurations of classical models and green functions of a quantum model), and it also performs well in detecting Kosterlitz-Thouless phase transitions.
BibTeX
@inproceedings{ding2022report,
  author = {Jiewei Ding and Ho-Kin Tang and Wing Chi Yu},
  title = {{Report on scipost\_202203\_00027v1}},
  booktitle = {Openalex Unknown},
  year = {2022},
  doi = {10.21468/scipost.report.4988},
}
Figure