Synthesizing controlled microstructures of porous media using generative adversarial networks and reinforcement learning

Phong Nguyen, Nikolaos N. Vlassis, Bahador Bahmani, WaiChing Sun, H. S. Udaykumar, Stephen BaekScientific 2022 · DOI

TL;DR

This paper introduces a novel deep learning approach combining Generative Adversarial Networks (GANs) and Actor-Critic (AC) reinforcement learning to synthesize realistic 3D microstructures of porous media with user-defined physical properties. This method overcomes limitations of previous approaches by enabling precise control over material attributes, paving the way for "materials-by-design" and accelerating material discovery.

What Problem Does This Paper Address?

Creating accurate digital twins of porous materials is crucial for material modeling and discovery. While direct numerical simulations (DNS) on synthetic microstructures can supplement costly physical experiments, synthesizing realistic microstructures with controlled physical properties remains a significant challenge. Existing methods often oversimplify microstructural attributes, leading to inaccurate representations of the physical world. Furthermore, previous GAN-based approaches for microstructure generation lacked precise control over desired physical quantities of interest (QoI) and were often limited to 2D or specific material types.

Methods

The core of this work is a novel deep learning framework that integrates a 3D Generative Adversarial Network (GAN) with an Actor-Critic (AC) reinforcement learning agent.

  • 3D GAN for Microstructure Generation: A modified 3D GAN architecture is employed to generate realistic 3D microstructures. Key modifications include reducing convolutional blocks for efficiency, adding stride-1 convolutional layers to enhance expressiveness, and incorporating batch normalization for training stability. The generator takes global morphology parameters and local stochasticity parameters as input to control the microstructure's appearance.
  • Actor-Critic (AC) Reinforcement Learning for Control: An AC reinforcement learning agent is used to tune the morphology parameters of the 3D GAN.
  • Actor: This network learns a policy to adjust the GAN's input parameters (morphology parameters) to achieve targeted physical properties.
  • Critic: This network evaluates the quality of the generated microstructures by comparing their physical properties to the desired targets, providing feedback to the actor.
  • The AC agent learns through trial-and-error, iteratively refining its policy to generate microstructures that match user-defined QoIs. The process involves generating a microstructure, estimating its properties, calculating a reward based on the difference from the target, and updating the actor and critic networks.

Datasets

The study utilizes micro computed tomography (µCT) scans of Bentheim sandstone as the training and validation dataset. Specifically, 256x256x256 sub-volumes were sampled from original 704x678x500 data and then coarsened. The paper also mentions making 10 synthetic 3D microstructures available via an open-access data repository.

Key Insights and Ideas

  • Synergy of GANs and RL for Controlled Synthesis: The combination of GANs for realistic generation and AC reinforcement learning for precise control is a powerful paradigm for inverse material design. This approach allows for the generation of microstructures that are not only visually convincing but also possess specific, user-defined physical properties.
  • Addressing 3D Complexity: Extending GANs to 3D microstructures presents significant challenges due to increased dimensionality and complexity. The paper's architectural modifications and training strategies effectively tackle these issues, enabling the generation of high-quality 3D porous media.
  • Beyond Visual Realism: While visual realism is important, the true innovation lies in controlling physical properties. The AC agent's ability to learn the complex mapping between morphology parameters and physical quantities (like porosity and permeability) is a key breakthrough.
  • "Materials-by-Design" Enabled: This work moves closer to the concept of "materials-by-design" by providing a tool that can synthesize materials with desired properties, rather than just analyzing existing ones.

Key Results

  • Visual Realism: The generated 3D microstructures are visually indistinguishable from real specimens, with diverse and realistic void and crystal shapes.
  • Quantitative Property Control: The GAN-AC framework achieves targeted physical properties with an average error of 2.54% for porosity, 3.51% for specific surface area, and 4.06% for effective permeability on a test set of 500 examples.
  • Broader Property Range: The generated microstructures exhibit porosities that cover a larger range (0.1 to 0.25) compared to the ground truth (0.18 to 0.25), suggesting potential for extending material databases.
  • Efficiency Compared to Bayesian Optimization: The GAN-AC method is significantly more efficient than Bayesian Optimization (BO) in terms of objective function evaluations, requiring an average of 8.7 evaluations compared to BO's 23.7 (uniform sampling) and 24.4 (LHS sampling) for similar solution quality.

Limitations and Future Work

  • Computational Cost: While efficient in inference, the training of the GAN-AC framework can be computationally demanding.
  • Generalizability: The current model is trained for a specific type of material (Bentheim sandstone). For different materials or significantly different target QoIs, retraining of the AC model is required.
  • Future Work Suggestions:
    • Multi-agent AC: Exploring multi-agent AC models with decentralized policies to accelerate exploration.
  • Uncertainty Quantification: Incorporating aleatoric and epistemic uncertainties to account for stochasticity in real-world materials.
  • Manufacturability: Including manufacturability constraints and process-structure relationships into the design process.
  • Physics-Informed ML: Designing neural network architectures that inherently satisfy relevant physical constraints (e.g., material symmetry).

Summary updated

Abstract

For material modeling and discovery, synthetic microstructures play a critical role as digital twins. They provide stochastic samples upon which direct numerical simulations can be conducted to populate material databases. A large ensemble of simulation data on synthetic microstructures may provide supplemental data to inform and refine macroscopic material models, which might not be feasible from physical experiments alone. However, synthesizing realistic microstructures with realistic microstructural attributes is highly challenging. Thus, it is often oversimplified via rough approximations that may yield an inaccurate representation of the physical world. Here, we propose a novel deep learning method that can synthesize realistic three-dimensional microstructures with controlled structural properties using the combination of generative adversarial networks (GAN) and actor-critic (AC) reinforcement learning. The GAN-AC combination enables the generation of microstructures that not only resemble the appearances of real specimens but also yield user-defined physical quantities of interest (QoI). Our validation experiments confirm that the properties of synthetic microstructures generated by the GAN-AC framework are within a 5% error margin with respect to the target values. The scientific contribution of this paper resides in the novel design of the GAN-AC microstructure generator and the mathematical and algorithmic foundations therein. The proposed method will have a broad and substantive impact on the materials community by providing lenses for analyzing structure-property-performance linkages and for implementing the notion of 'materials-by-design'.
BibTeX
@inproceedings{nguyen2022synthesizing,
  author = {Phong Nguyen and Nikolaos N. Vlassis and Bahador Bahmani and WaiChing Sun and H. S. Udaykumar and Stephen Baek},
  title = {{Synthesizing controlled microstructures of porous media using generative adversarial networks and reinforcement learning}},
  booktitle = {Scientific},
  year = {2022},
  doi = {10.1038/s41598-022-12845-7},
}
Figure