Introduction
The Ising model serves as a fundamental framework in statistical mechanics for understanding ferromagnetism and phase transitions. While the 2D Ising model is analytically solvable, more complex variations often require computationally intensive numerical methods. Recently, Machine Learning (ML) has emerged as a powerful tool for identifying phase transitions. Traditional approaches have largely relied on Convolutional Neural Networks (CNNs). However, CNNs possess inherent inductive biases, such as translation invariance, which may limit their ability to capture the long-range global correlations critical to physical systems near a phase transition. This study investigates the application of Vision Transformers (ViTs) to predict state variables in 2D Ising models, leveraging their attention mechanisms to overcome the limitations of local feature extraction.
Methodology and Dataset Generation
The researchers utilized a ViT-Base model, pre-trained on ImageNet-21k and fine-tuned on ImageNet-1k, followed by domain-specific fine-tuning on Ising model microstates. Unlike CNNs, which use sliding windows to process local features, ViTs decompose images into sequences of flattened patches. Self-attention mechanisms then allow the model to weigh the relative importance of these patches across the entire image, regardless of spatial distance.
To train and evaluate the models, a custom dataset of 100x100 lattice images was generated using the Metropolis-Hastings algorithm. The dataset included 1,300 images for each of three boundary conditions: Periodic, Anti-periodic, and Skewed ±. These microstates were categorized into four temperature-based bins representing different physical regimes:
Physical States and Phase Transitions
The model's task was to classify these microstates across various physical configurations, including both ferromagnetic and anti-ferromagnetic couplings. At low temperatures ($T=0$), ferromagnetic systems exhibit uniform alignment (all yellow or all blue), while anti-ferromagnetic systems form a perfect checkerboard pattern. As temperature increases ($T \gg T_c$), both systems transition into a disordered, high-entropy state.
Key Results and Performance
The performance of the ViT was compared against ResNet-18 and ResNet-50 baselines using Macro F1 scores. Remarkably, the ViT consistently outperformed the CNNs across all boundary conditions, even when trained on relatively small datasets. This suggests that the general features learned during ImageNet pre-training provide a robust foundation for physical state classification.
| System/Boundary Condition | ViT (F1) | ResNet-50 (F1) |
|---|---|---|
| Periodic (Ferromagnetic) | 0.934 | 0.907 |
| Periodic (Anti-ferromagnetic) | 0.935 | 0.899 |
| Skewed (Ferromagnetic) | 0.931 | 0.917 |
| Anti-Periodic (Ferromagnetic) | 0.921 | 0.917 |
The ViT demonstrated particular strength in the "Critical Region" (CR), a narrow temperature band characterized by high noise and subtle features, which is traditionally the most difficult region for classification.
Conclusion and Future Directions
This research highlights the potential of Vision Transformers in the field of computational physics. Beyond superior accuracy, ViTs offer enhanced interpretability through attention maps, which can reveal the specific lattice regions the model prioritizes during classification. Future work will extend this approach to more complex systems, such as the XY and q-Potts models, and explore the use of ViTs in discovering novel topological phase transitions.
Summary updated