Integration of AI and mechanistic modeling in generative adversarial networks for stochastic inverse problems

Jaimit Parikh, James R. Kozloski, Viatcheslav GurevarXiv 2020 · arXiv

Introduction

Stochastic Inverse Problems (SIPs) are fundamental to scientific discovery in fields ranging from systems biology to geophysics. The goal is to determine the distribution of input parameters for a mechanistic model (MM) that best explains an observed distribution of experimental data. Unlike traditional parameter estimation, which focuses on individual instances, SIPs seek to characterize the statistical properties of an entire ensemble.

However, existing approaches face significant challenges. Bayesian methods often introduce "prior bias," where the choice of prior distribution unduly influences the results. Meanwhile, data-consistent methods like rejection sampling suffer from the "curse of dimensionality" and struggle to incorporate internal stochastic noise within the mechanistic model. This paper introduces the Regularized GAN (r-GAN), an optimization-based framework that leverages deep learning to solve SIPs efficiently and accurately.

The Regularized GAN (r-GAN) Framework

The core innovation of this work is the reformulation of the SIP as a constrained optimization problem within a Generative Adversarial Network (GAN) architecture. Standard GANs consist of a generator and a single discriminator. The r-GAN extends this by incorporating the mechanistic model directly into the training loop and adding a second discriminator.

Schematic of the r-GAN architecture featuring dual discriminators for parameter and output consistency.

As shown in the architecture diagram, the r-GAN utilizes two distinct discriminators:

  1. Output Discriminator ($D_Y$): This component ensures that the distribution of outputs produced by the mechanistic model ($Q_{Y_g}$) matches the target experimental data distribution ($Q_Y$).
  2. Parameter Discriminator ($D_X$): This acts as a regularizer. It ensures that the generated parameter distribution ($Q_{X_g}$) remains consistent with the physical constraints defined by the parameter prior ($\mathcal{P}_X$), without imposing the rigid bias typical of Bayesian posteriors.

By minimizing the Jensen-Shannon Divergence (JSD) across both discriminators, the generator learns to produce parameter sets that are both physically plausible and capable of recreating observed phenomena.

Handling Stochasticity and Model Surrogates

A major strength of the r-GAN is its ability to handle Stochastic Simulators. In many real-world scenarios, the mechanistic model itself contains random noise (e.g., measurement error). The authors demonstrate that r-GAN can treat noise parameters as trainable variables, allowing the system to "learn the noise" alongside the physical parameters. This prevents the bias that occurs in traditional methods when noise and parameters are correlated.

To maintain efficiency, the r-GAN requires the mechanistic model to be differentiable. For non-differentiable models, the authors employ surrogate modeling, training a neural network to mimic the MM's behavior. This allows the gradient-based optimization of the GAN to proceed even when the underlying physics engine is a "black box."

High-Dimensional Performance: MNIST Super-Resolution

To test the scalability of the r-GAN, the authors applied it to a high-dimensional MNIST super-resolution task. Here, the "mechanistic model" is a blurring function (average pooling). The goal is to recover the distribution of high-resolution digits (parameters) from a distribution of low-resolution observations.

Comparison of MNIST super-resolution results between standard GAN and r-GAN.

The results highlight the limitations of standard GANs in this context. While a standard GAN might produce a diverse set of digits, it fails to capture the specific distribution of the target data (e.g., a population consisting only of the digit "5"). In contrast, the r-GAN successfully recovers the target distribution, achieving a JSD of 0.22 compared to 0.73 for the standard GAN. This demonstrates the framework's ability to solve complex inverse problems where the parameter space is high-dimensional and multi-modal.

Conclusion

The r-GAN framework represents a significant step forward in solving Stochastic Inverse Problems. By combining the interpretability of mechanistic models with the representational power of GANs, it offers a scalable, data-consistent alternative to traditional sampling methods. While challenges remain—such as ensuring differentiability through surrogates and preventing mode collapse—the r-GAN's ability to handle stochasticity and high-dimensional data makes it a promising tool for modern scientific inference.

Summary updated

BibTeX
@misc{parikh2020integration,
  author = {Jaimit Parikh and James R. Kozloski and Viatcheslav Gurev},
  title = {{Integration of AI and mechanistic modeling in generative adversarial networks for stochastic inverse problems}},
  year = {2020},
  eprint = {2009.08267},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2009.08267},
}
Figure