Feature Disentanglement in generating three-dimensional structure from two-dimensional slice with sliceGAN

Hyungjin Chung, Jong Chul YearXiv 2021 · arXiv

Abstract

This paper presents an enhancement to sliceGAN, a generative model for reconstructing 3D material microstructures from 2D slices. By incorporating Adaptive Instance Normalization (AdaIN) and a dedicated code generator, the authors achieve feature disentanglement, allowing for continuous control over physical properties such as grain size during inference. This approach eliminates the need for retraining the model for every structural variation, significantly reducing computational overhead while maintaining high fidelity in generated volumes. The model demonstrates successful interpolation between discrete training points, though it faces limitations in extrapolation beyond the training bounds.

Introduction

In materials science, 3D microstructures are essential for accurate simulations of material behavior. However, obtaining high-resolution 3D data is often prohibitively expensive and time-consuming compared to acquiring 2D microscopic slices. The original sliceGAN model addressed this by using Generative Adversarial Networks (GANs) to learn the micro-structural distribution of a 2D slice and generate representative 3D volumes.

Despite its success, the original sliceGAN lacks control over the generated output. It maps a uniform random noise vector to a 3D volume, meaning specific visual or physical attributes cannot be steered. To generate materials with different grain sizes, researchers previously had to train a separate model for every variation—a process requiring roughly 4 hours of training time per dataset. This paper introduces a method to disentangle the latent space, enabling continuous control over specific features within a single trained model.

Methods

The authors adapt the concept of Adaptive Instance Normalization (AdaIN), popularized by StyleGAN, to the domain of materials science. The standard sliceGAN architecture, which utilizes a 3D generator and three 2D discriminators (evaluating $x$, $y$, and $z$ planes), is modified by replacing standard batch normalization layers with AdaIN blocks.

Architecture of the modified sliceGAN with AdaIN blocks and code generator.

A key addition is the Multi-Layer Perceptron (MLP) "code generator." This MLP takes a static scalar value representing a specific feature—such as a scale parameter for grain size—and converts it into a style vector. The AdaIN layers then inject this style vector into the generator after every convolution. This mechanism disentangles the feature space: while random noise dictates the general stochastic structure, the injected code dictates the specific physical attribute (e.g., grain size).

Experimental Setup

The proof-of-concept utilized an open-source database of synthetic 3D microstructures generated via DREAM3D. The data consists of three-phase materials with volumes of $75 \times 75 \times 15 \mu m^3$ and a 125 nm voxel size. To evaluate feature control, the authors selected three datasets with varying scale parameters (log of geometric standard deviation, $\ln \sigma_g$) set to 0.15, 0.35, and 0.60, representing small, medium, and large grain sizes respectively.

Key Results

The primary finding is that the model learns a continuous representation of the feature space from discrete training data. By training on only three distinct grain sizes, the model successfully understands the spectrum between them.

Training data patches and continuous interpolation results for varying grain sizes.

At inference time, the model was fed scale parameters ($\alpha$) it had never encountered during training (e.g., 0.175, 0.225, 0.300, 0.400). As shown in the results, the model synthesized highly realistic 3D volumes and 2D cut-views with grain sizes that perfectly matched these intermediate values. This continuous latent space offers massive efficiency gains, as a single model can synthesize an infinite variety of grain sizes within its trained range.

However, the model's ability to generalize is limited to interpolation. Attempts at extrapolation failed; inputting values smaller than 0.15 or larger than 0.60 did not yield corresponding changes in grain size, indicating the model cannot reliably generate features outside its training bounds.

Conclusion and Future Work

This work demonstrates that style transfer techniques like AdaIN are highly effective at disentangling physical micro-structural properties. While successful for discrete $n$-phase materials, modeling continuous grayscale data remains a challenge for future research. The authors suggest that scaling up the generator and discriminator architectures and implementing progressive growing strategies could improve stability and output quality for more complex volumes, such as RGB color data, though these improvements will require increased computational resources.

Summary updated

Abstract

Deep generative models are known to be able to model arbitrary probability distributions. Among these, a recent deep generative model, dubbed sliceGAN, proposed a new way of using the generative adversarial network (GAN) to capture the micro-structural characteristics of a two-dimensional (2D) slice and generate three-dimensional (3D) volumes with similar properties. While 3D micrographs are largely beneficial in simulating diverse material behavior, they are often much harder to obtain than their 2D counterparts. Hence, sliceGAN opens up many interesting directions of research by learning the representative distribution from 2D slices, and transferring the learned knowledge to generate arbitrary 3D volumes. However, one limitation of sliceGAN is that latent space steering is not possible. Hence, we combine sliceGAN with AdaIN to endow the model with the ability to disentangle the features and control the synthesis.
BibTeX
@misc{chung2021feature,
  author = {Hyungjin Chung and Jong Chul Ye},
  title = {{Feature Disentanglement in generating three-dimensional structure from two-dimensional slice with sliceGAN}},
  year = {2021},
  eprint = {2105.00194},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2105.00194},
}
Figure