Abstract
We provide and critically analyze a framework to learn critical behavior in classical partition functions through the application of non-parametric methods to data sets of thermal configurations. We illustrate our approach in phase transitions in 2D and 3D Ising models. First, we extend previous studies on the intrinsic dimension of 2D partition function data sets, by exploring the effect of volume in 3D Ising data. We find that as opposed to 2D systems for which this quantity has been successfully used in unsupervised characterizations of critical phenomena, in the 3D case its estimation is far more challenging. To circumvent this limitation, we then use the principal component analysis (PCA) entropy, a "Shannon entropy" of the normalized spectrum of the covariance matrix. We find a striking qualitative similarity to the thermodynamic entropy, which the PCA entropy approaches asymptotically. The latter allows us to extract—through a conventional finite-size scaling analysis with modest lattice sizes—the critical temperature with less than 1% error for both 2D and 3D models while being computationally efficient.
Introduction
In statistical mechanics, identifying phase transitions typically requires knowledge of an order parameter. However, for many complex systems, this parameter is unknown. Unsupervised machine learning offers a path forward by detecting transitions directly from raw data snapshots. This report examines a framework using Intrinsic Dimension ($I_d$) and PCA Entropy ($S_{PCA}$) to characterize criticality in the 2D and 3D Ising models.
The Challenge of Dimensionality in 3D
Previous research successfully utilized the intrinsic dimension—the minimum number of variables needed to describe a dataset—to identify transitions in 2D systems. In 2D, $I_d$ typically exhibits a distinct "dip" at the critical point. However, extending this to 3D systems reveals significant hurdles.
As seen in Figure 1, the local $I_d$ estimator (TWO-NN) in the 3D Ising model is highly sensitive to lattice size and noise. The "curse of dimensionality" in 3D results in a complex, curved data manifold that local distance-based methods struggle to map accurately, leading to unreliable estimates of the critical temperature.
PCA-Based Robustness and Entropy
To address these limitations, the authors propose global estimators based on Principal Component Analysis (PCA). Two primary measures are explored:
- PCA-based $I_d$ ($I_d^{PCA}$): A global estimator that counts the number of components required to explain a certain percentage of the data's variance.
- PCA Entropy ($S_{PCA}$): The Shannon entropy of the normalized eigenvalues of the covariance matrix.
$S_{PCA}$ is particularly noteworthy for its qualitative similarity to thermodynamic entropy. It remains low in the ordered phase, where a few dominant configurations exist, and increases as the system enters the disordered phase.
Figure 4 demonstrates that PCA-based global measures are far more robust than local $I_d$ estimators. By applying finite-size scaling (FSS) to the derivative of these curves, the authors estimated the critical temperature for the 3D Ising model at $T_c \approx 4.518$, achieving less than 1% error relative to the literature value of ~4.51.
Conclusion
The introduction of PCA entropy provides a powerful, computationally efficient tool for unsupervised physics. By bridging information theory and statistical mechanics, this framework allows for the precise detection of phase transitions without prior knowledge of the Hamiltonian or order parameters. Future applications include exploring topological transitions (like BKT) and extending the method to quantum many-body systems.
Summary updated