Data-Targeted Prior Distribution for Variational AutoEncoder

Nissrine Akkari, Fabien Casenave, Thomas Daniel, David RyckelynckFluids 2021 · DOI

TL;DR

This paper introduces a novel "data-targeted" prior distribution for Variational Autoencoders (VAEs) applied to fluid dynamics simulations. By leveraging Kernel Proper Orthogonal Decomposition (KPOD) coefficients from training data, their VAE generates more accurate and physically plausible velocity fields, improving the initialization of complex simulations like Large Eddy Simulations (LES).

What Problem Does This Paper Address?

Initializing complex fluid dynamics simulations, such as Large Eddy Simulations (LES) for turbulent flows in aircraft engines, is computationally expensive and crucial for accuracy. Traditional methods often struggle with the high dimensionality and complexity of these flows. Variational Autoencoders (VAEs) offer a promising approach for learning latent representations of these flows, but their performance is heavily influenced by the choice of prior distribution for the latent variables. The standard practice of using a simple normal distribution as a prior can limit the VAE's ability to accurately capture the true posterior distribution of the flow dynamics, leading to less accurate reconstructions and generations. This paper aims to address this limitation by proposing a more informed, data-driven prior distribution.

Methods

The core of this work is the development of a "data-targeted" prior distribution for VAEs. Instead of the conventional standard normal prior, the authors propose using the coefficients obtained from Kernel Proper Orthogonal Decomposition (KPOD) of the training data. KPOD is a non-linear dimensionality reduction technique that captures the essential dynamics of the flow in a feature space.

The authors then integrate this data-targeted prior into a VAE framework. A key innovation is a modified training algorithm that penalizes the difference between the VAE's latent variable and the KPOD coefficients. This approach avoids the need for a complex reparametrization trick for the new prior, allowing it to be compatible with various VAE architectures. The VAE's encoder learns to approximate the posterior distribution of the flow, while the decoder reconstructs the physical velocity fields.

Datasets

The study utilizes a 2D unsteady, compressible, reacting fluid flow test case from an aeronautical injector simulation. The high-fidelity data for training and evaluation were generated using the YALES2 flow solver. The training set consists of 998 snapshots of velocity and pressure fields extracted from 5000 time steps of a high-fidelity simulation.

Key Insights and Ideas

  • Prior Matters: The choice of prior distribution in VAEs significantly impacts their ability to accurately model complex physical phenomena like turbulent flows. A generic prior (like the standard normal) is insufficient for capturing the intricate dynamics.
  • KPOD as a Data-Driven Prior: KPOD coefficients provide a rich, low-dimensional representation of the flow's dynamics. Using these coefficients as a prior distribution for the VAE's latent space allows the model to learn more relevant and physically grounded representations.
  • Bridging Model Order Reduction and Deep Learning: This work effectively bridges the gap between traditional model order reduction techniques (like KPOD) and modern deep learning approaches (VAEs), leveraging the strengths of both.
  • Interpretable Latent Space: The data-targeted prior leads to a more interpretable latent space where the parameterized mean and variance of the VAE's latent representation are directly related to the underlying physical dynamics captured by KPOD.
  • Improved Generative Capabilities: The data-targeted VAE demonstrates superior performance in generating physically plausible velocity fields that better satisfy conservation laws (like mass conservation) compared to a standard VAE.

Key Results

  • The proposed data-targeted VAE shows improved reconstruction of high-fidelity velocity and pressure fields compared to a classical VAE with a standard normal prior. This is visually demonstrated in Figures 7-12, where the reconstructed fields from the data-targeted VAE are closer to the ground truth.
  • The Kullback-Leibler (KL) divergence between the VAE's latent distribution and the standard normal prior does not converge for the classical VAE, indicating a poor fit. In contrast, the data-targeted VAE's approach, while not directly optimizing for KL divergence with a standard normal prior, leads to better overall performance.
  • Generated velocity fields from the data-targeted VAE exhibit significantly better adherence to the mass conservation equation compared to those generated by the classical VAE (Figures 20-22). This is a crucial indicator of physical realism in fluid dynamics.

Limitations and Future Work

  • Computational Cost of KPOD: While KPOD provides a valuable prior, its computation can be resource-intensive, especially for very large datasets.
  • Generalizability to Other Flows: The study focuses on a specific turbulent reacting flow in an aeronautical injector. Further work is needed to assess the generalizability of this approach to other types of fluid flows and physical phenomena.
  • Direct Application to LES Initialization: The authors explicitly state that a key future direction is the direct application of this methodology for turbulence injection problems in Large Eddy Simulations. This would involve using the VAE to initialize LES runs, potentially leading to significant computational savings and improved accuracy.
  • Exploring Other Kernel Functions: The study primarily uses a Gaussian kernel for KPOD. Investigating other kernel functions could potentially yield further improvements.

Summary updated

Abstract

Bayesian methods were studied in this paper using deep neural networks. We are interested in variational autoencoders, where an encoder approaches the true posterior and the decoder approaches the direct probability. Specifically, we applied these autoencoders for unsteady and compressible fluid flows in aircraft engines. We used inferential methods to compute a sharp approximation of the posterior probability of these parameters with the transient dynamics of the training velocity fields and to generate plausible velocity fields. An important application is the initialization of transient numerical simulations of unsteady fluid flows and large eddy simulations in fluid dynamics. It is known by the Bayes theorem that the choice of the prior distribution is very important for the computation of the posterior probability, proportional to the product of likelihood with the prior probability. Hence, we propose a new inference model based on a new prior defined by the density estimate with the realizations of the kernel proper orthogonal decomposition coefficients of the available training data. We numerically show that this inference model improves the results obtained with the usual standard normal prior distribution. This inference model was constructed using a new algorithm improving the convergence of the parametric optimization of the encoder probability distribution that approaches the posterior. This latter probability distribution is data-targeted, similarly to the prior distribution. This new generative approach can also be seen as an improvement of the kernel proper orthogonal decomposition method, for which we do not usually have a robust technique for expressing the pre-image in the input physical space of the stochastic reduced field in the feature high-dimensional space with a kernel inner product.
BibTeX
@inproceedings{akkari2021data,
  author = {Nissrine Akkari and Fabien Casenave and Thomas Daniel and David Ryckelynck},
  title = {{Data-Targeted Prior Distribution for Variational AutoEncoder}},
  booktitle = {Fluids},
  year = {2021},
  doi = {10.3390/fluids6100343},
}
Figure