Abstract: We found that Bidirectional LSTM and Transformer can classify different phases of condensed matter models and determine the phase transition points by learning features in the Monte Carlo raw data before equilibrium. Our method can significantly reduce the time and computational resources required for probing phase transitions as compared to the conventional Monte Carlo simulation. We also provide evidence that the method is robust and the performance of the deep learning model is insensitive to the type of input data (we tested spin configurations of classical models and green functions of a quantum model), and it also performs well in detecting Kosterlitz-Thouless phase transitions.
Introduction
In condensed matter physics, identifying phase transitions often relies on Monte Carlo (MC) simulations. A significant challenge is the "critical slowing down" near transition points, where the time required for a system to reach equilibrium increases dramatically. In quantum systems, the "sign problem" further exacerbates computational costs. This research proposes a novel approach: using deep learning to predict phase transitions from raw MC data before equilibrium is reached.
Methods
The core innovation treats MC simulation data as a time-series problem. Instead of analyzing full spatial configurations, the researchers sample a small number of sites and track their evolution over the first 10 to 20 MC steps.
The study compared several architectures:
- Bidirectional LSTM (Bi-LSTM): Captures temporal correlations by processing sequences in both forward and backward directions.
- Transformer: Utilizes self-attention mechanisms to weigh the importance of different steps in the sequence.
- Benchmarks: Fully Connected (FCN) and Convolutional Neural Networks (CNN) were used as baselines but generally underperformed in handling sequential data.
Datasets and Evaluation
The models were tested on diverse physical systems:
- Classical Ising Model: Square, honeycomb, and triangular lattices.
- XY Model: Known for topological Kosterlitz–Thouless (KT) transitions.
- Quantum Hubbard Model: Using Green functions as input data.
Phase transition points ($T_c$) were identified as the temperature where the model's classification confidence between two phases was 50%.
Key Insights
- Temporal over Spatial: Bi-LSTMs and Transformers effectively extract the "DNA" of the final equilibrium state from early temporal evolution, outperforming spatial-focused CNNs.
- Markovian Simplicity: For MC processes, positional encoding in Transformers proved unnecessary, as the Markovian nature of the data makes the specific order of early steps less critical than the overall trajectory.
- Topological Sensitivity: The models successfully identified KT transitions without explicit spatial information, suggesting they can learn complex topological features from sparse temporal samples.
Key Results
The proposed method offers a staggering reduction in computational overhead. For a $128 \times 128$ Ising model, the deep learning approach achieved a 77% reduction in time compared to traditional order parameter calculations. When compared to supervised learning models that require fully equilibrated data, the speedup reached 99%.
Crucially, Bi-LSTM and Transformers maintained high accuracy using only 10–20 MC steps, whereas traditional methods often require over 500 steps to reach equilibrium. While the quantum Hubbard model showed some output fluctuations near $T_c$, the transition points were still correctly identified.
Conclusion
This work demonstrates that deep learning models, particularly those designed for sequence processing, can bypass the equilibrium bottleneck in Monte Carlo simulations. By learning from the pre-equilibrium trajectory, these models provide a robust and highly efficient tool for probing phase transitions in both classical and quantum systems. Future research may extend this to real-time dynamics in molecular simulations or glass transitions.
Summary updated