DeepThermo: Deep Learning Accelerated Parallel Monte Carlo Sampling for Thermodynamics Evaluation of High Entropy Alloys

Junqi Yin, Feiyi Wang, Mallikarjun Arjun Shankar

Introduction

Understanding the thermodynamic properties of complex materials, such as High Entropy Alloys (HEAs), is a fundamental challenge in materials science. HEAs, which consist of five or more elements, possess an astronomically large configuration space that scales as $O(M^N)$. Traditional Monte Carlo (MC) sampling methods, such as Metropolis or Parallel Tempering, often struggle to navigate this space. At low temperatures, the energy landscape becomes rugged, and local proposals—like swapping two atoms—are frequently insufficient to overcome energy barriers, causing simulations to become trapped in local minima.

While Wang-Landau (WL) sampling offers a solution by sampling energy states directly, it is notoriously difficult to parallelize effectively. The sequential nature of standard WL algorithms prevents them from fully utilizing modern High-Performance Computing (HPC) platforms. This paper introduces DeepThermo, a framework that integrates traditional physics simulations with deep learning to enable massively parallel thermodynamic evaluations.

The DeepThermo Framework

DeepThermo overcomes the limitations of local MC updates by introducing a Deep Learning-Generated MC Proposal. Instead of incremental swaps, the framework uses a 3D Variational Autoencoder (VAE) to propose global updates to the entire atomic grid. The methodology follows a three-stage workflow:

The three-stage DeepThermo workflow: initial data collection via Parallel Tempering, VAE training, and final parallel Wang-Landau sampling.
  1. Data Collection: A short Parallel Tempering (PT) simulation is conducted to gather a diverse set of atomic configurations, ranging from ordered to disordered states.
  2. VAE Training: A 3D VAE is trained on these configurations, learning to compress the complex physical grid into a low-dimensional "latent space."
  3. Parallel WL Sampling: During the main simulation, the system performs random walks within the VAE's latent space. These points are decoded back into 3D configurations, providing global jumps that align with the energy-flatness goals of Wang-Landau sampling.

Latent Space and Global Proposals

The core innovation of DeepThermo lies in the organization of the VAE's latent space. The model naturally clusters configurations based on their degree of order, which correlates directly with system energy.

Architecture of the 3D Variational Autoencoder (VAE) used to generate global Monte Carlo proposals by navigating a low-dimensional latent space.

By navigating this latent space, the sampler can bypass massive physical energy barriers that would stall traditional local-swap algorithms. This global update mechanism allows parallel WL samplers to remain decoupled and scale across thousands of GPUs without getting trapped in distorted energy regions. To ensure high performance, the framework is hardware-agnostic, supporting both NVIDIA (CUDA) and AMD (ROCm) platforms, and utilizes optimized inference backends like TensorFlow C++ and SmartRedis.

Key Results and Physical Discovery

DeepThermo has demonstrated unprecedented scale and efficiency, successfully running on up to 3,072 GPUs on the Summit and Crusher supercomputers. The framework achieved a weak scaling efficiency of ~78% and a strong scaling efficiency of ~67%.

This computational power enabled the researchers to map the density of states for the MoNbTaW alloy over a range of $\sim e^{10,000}$, a feat previously considered impractical. More importantly, the high-resolution sampling led to a new physics discovery: while the known order-disorder transition near 900K was confirmed, the framework also identified a newly observed second-order phase transition (partial-order to order) near 300K.

Limitations and Future Work

Despite its success, DeepThermo faces challenges related to information loss during dimensionality reduction in the VAE, which can lead to slight underestimations of the density of states. Currently, the frequency of deep learning proposals must be reduced as the simulation converges to maintain mathematical accuracy. Future work will focus on automating the end-to-end pipeline, integrating the three stages into a seamless, turnkey solution for thermodynamic sampling on HPC systems.

Summary updated

Abstract

Since the introduction of Metropolis Monte Carlo (MC) sampling, it and its variants have become standard tools used for thermodynamics evaluations of physical systems. However, a long-standing problem that hinders the effectiveness and efficiency of MC sampling is the lack of a generic method (a.k.a. MC proposal) to update the system configurations. Consequently, current practices are not scalable. Here we propose a parallel MC sampling framework for thermodynamics evaluation—DeepThermo. By using deep learning–based MC proposals that can globally update the system configurations, we show that DeepThermo can effectively evaluate the phase transition behaviors of high entropy alloys, which have an astronomical configuration space. For the first time, we directly evaluate a density of states expanding over a range of ~e<sup>10,000</sup> for a real material. We also demonstrate DeepThermo’s performance and scalability up to 3,000 GPUs on both NVIDIA V100 and AMD MI250X-based supercomputers.
BibTeX
@inproceedings{yin2023deepthermo,
  author = {Junqi Yin and Feiyi Wang and Mallikarjun Arjun Shankar},
  title = {{DeepThermo: Deep Learning Accelerated Parallel Monte Carlo Sampling for Thermodynamics Evaluation of High Entropy Alloys}},
  booktitle = {IPDPS},
  year = {2023},
  doi = {10.1109/ipdps54959.2023.00041},
}
Figure