Identifying phase transitions in physical systems with neural networks: a neural architecture search perspective

Rodrigo Carmo Terin, Zochil González Arenas, Roberto SantanaarXiv 2024 · DOI · arXiv

Abstract

Identifying phase transitions in condensed-matter physics is a fundamental challenge, often complicated by the absence of well-defined order parameters in complex systems. While neural networks have shown promise in classifying phases, their architectural design remains largely heuristic, and their internal decision-making processes are often opaque. This study addresses these challenges by combining Neural Architecture Search (NAS) with software-testing metrics—specifically "neuron coverage"—to analyze the 2D Classical Ising Model. We employ a neuroevolutionary algorithm to automate the discovery of optimal Multi-Layer Perceptron (MLP) structures for temperature prediction. Our findings reveal that the internal activation patterns of these evolved networks, measured via neuron coverage (NC), act as sensitive indicators of physical phase transitions. Specifically, mean NC peaks at the critical temperature ($T_c$), providing a novel, automated proxy for identifying points of physical disorder.

Introduction

In the study of physical systems, phase transitions represent critical points where a system's macroscopic properties undergo abrupt changes. Traditional methods for identifying these transitions rely on order parameters, which may be difficult to define for novel or complex materials. Machine learning, particularly deep learning, has emerged as a powerful alternative. However, two primary hurdles persist: the "Architecture Problem," where network design is often arbitrary, and the "Black Box Problem," where the physical basis for a network's prediction remains hidden. This work proposes an integrated approach using automated architecture search and internal state analysis to bridge the gap between machine learning performance and physical interpretability.

Methodology

The research utilizes a two-pronged approach:

  1. Neuroevolutionary NAS: Instead of manual design, a genetic algorithm is used to evolve MLP architectures. The search space includes the number of layers, neurons per layer, and activation functions. The fitness function is based on the network's ability to predict the temperature of a physical system from its configuration.
  2. Neuron Coverage (NC) Metrics: Borrowed from software engineering, NC measures the proportion of a network's neurons that are activated above a specific threshold for a given input. We investigate whether these internal metrics correlate with the physical state of the system, particularly during phase transitions.

The methodology was tested on the 2D Classical Ising Model, a benchmark for ferromagnetism. Spin configurations (8x8 grids) were generated using Markov Chain Monte Carlo (MCMC) simulations across 26 temperatures ranging from 1.0 K to 3.5 K. The dataset comprised 65,000 samples split into training, validation, and test sets.

Results and Discussion

The NAS procedure successfully identified high-performing architectures, with over 95% of evolved models achieving a Test Mean Squared Error (MSE) below 0.15. Analysis of the prediction error across the temperature range revealed a characteristic "W-shape" signature.

Mean MSE as a function of temperature, exhibiting the characteristic W-shape with a peak at the critical temperature $T_c \approx 2.26K$.

As shown in the MSE analysis, the error peaks at the critical temperature ($T_c \approx 2.26K$) and at the temperature extremes. This indicates that the transition region, characterized by maximum fluctuations and disorder, is inherently the most challenging for the network to model accurately.

Beyond external performance, we analyzed the internal states of the networks using the Mean Neuron Coverage (NC) metric. This metric measures how many neurons are "firing" when presented with specific physical configurations.

Mean Neuron Coverage (NC) across temperatures, showing a distinct peak near the critical phase transition point.

As illustrated above, the Mean NC exhibits a clear peak near the critical temperature. This suggests that as the physical system enters a disordered state at the phase transition, the neural network utilizes a larger portion of its internal capacity to process the increased complexity of the input configurations. Furthermore, the standard deviation of the NC metric was found to reach its minimum near $T_c$, indicating that the network's internal response becomes remarkably stable and uniform when faced with the critical fluctuations of the phase transition.

Conclusion

This study demonstrates that automated architecture search can effectively replace manual design in physical machine learning tasks. More importantly, it shows that internal neural network metrics like neuron coverage can serve as automated indicators for physical phase transitions. While currently limited to small-scale classical models, this approach holds significant potential for identifying topological transitions in quantum systems and other complex many-body phenomena where traditional order parameters are unknown.

Summary updated

BibTeX
@misc{terin2024identifying,
  author = {Rodrigo Carmo Terin and Zochil González Arenas and Roberto Santana},
  title = {{Identifying phase transitions in physical systems with neural networks: a neural architecture search perspective}},
  year = {2024},
  eprint = {2404.15118},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.15118},
}
Figure