Reduced-order modeling of unsteady fluid flow using neural network ensembles

Rakesh Halder, Mohammadmehdi Ataei, Hesam Salehipour, Krzysztof Fidkowski, Kevin MakiarXiv 2024 · DOI · arXiv

Introduction

In the engineering domains of aerospace and marine design, high-fidelity Computational Fluid Dynamics (CFD) remains the gold standard for accuracy. However, the computational cost associated with these simulations is often prohibitive, requiring significant time and high-performance computing resources. To address this, researchers utilize Reduced-Order Models (ROMs)—surrogate models designed to approximate full-scale simulations at a fraction of the cost.

Despite their promise, data-driven ROMs face a significant hurdle when dealing with unsteady, time-varying fluid flows: error propagation. Most current models rely on an autoregressive approach, where the model predicts the next state based on its own previous outputs. In such systems, even minute errors at the start of a simulation can compound over hundreds of timesteps, leading to a phenomenon known as "drift," where the predicted flow eventually loses all physical resemblance to reality. This instability is particularly pronounced when the model is tasked with predicting "unseen" designs outside of its training data.

The CAE-eLSTM Framework

To combat this instability, a new framework called CAE-eLSTM has been developed. This non-intrusive, fully data-driven approach combines the spatial compression power of Convolutional Autoencoders (CAE) with the temporal stability of ensemble Long Short-Term Memory (LSTM) networks.

Spatial Compression via CAE

The first stage of the framework involves reducing the high-dimensional snapshots of fluid flow into a manageable format. A Convolutional Autoencoder (CAE) acts as the spatial compressor. It takes complex flow fields and squashes them into a low-dimensional "latent space" consisting of only a few key variables that represent the essential physics of the flow.

The architecture of the Convolutional Autoencoder (CAE) used to compress high-dimensional fluid snapshots into latent space.

Temporal Prediction and the Power of Ensembles

Once the spatial data is compressed, the model must predict how these latent variables evolve over time. While standard LSTMs are often used for this task, the CAE-eLSTM framework introduces a critical innovation: bagging (bootstrap aggregating).

Instead of relying on a single LSTM, the framework employs an ensemble of multiple LSTMs (ranging from 64 to 96 models depending on the complexity). Each LSTM in the ensemble is trained on a different random subset of the training data. During inference, the final prediction is the average of all individual model outputs.

The LSTM component predicts the next state (a_{t+1}) based on the sequence of previous latent states.

This ensemble averaging acts as a powerful stabilizer. While a single model might diverge due to specific weight initializations or noise in the training data, the collective average filters out these variances, keeping the simulation remarkably close to the physical ground truth over long durations.

Experimental Results and Performance

The framework was rigorously tested on two classic fluid dynamics problems: a 2D Lid-Driven Cavity simulation and a 2D Cylinder Flow (Von Kármán vortex street). In both cases, the CAE-eLSTM model demonstrated superior performance over single-model ROMs.

  • Accuracy and Robustness: The ensemble method significantly reduced relative $L^2$ errors and showed much lower standard deviation across different training runs, indicating high reproducibility.
  • Computational Speedup: The model achieved a 9.4x speedup for the lid-driven cavity and a staggering 27x speedup for the cylinder flow case compared to traditional CFD solvers like OpenFOAM and XLB.
  • Generalization: The model successfully predicted flow patterns for design parameters it had never seen during training, proving its utility as a robust engineering tool.

Conclusion and Future Directions

The CAE-eLSTM framework represents a significant step forward in making data-driven fluid simulations stable and reliable for long-term predictions. By leveraging ensemble methods to tame the drift inherent in autoregressive models, it offers a path toward real-time engineering analysis.

Future work will focus on extending this framework to 3D geometries and turbulent flows. Additionally, the researchers aim to move beyond structured grids by incorporating Graph Neural Networks (GNNs) to handle the complex, unstructured meshes common in industrial applications.

Summary updated

Abstract

The use of deep learning has become increasingly popular in reduced-order models (ROMs) to obtain low-dimensional representations of full-order models. Convolutional autoencoders (CAEs) are often used to this end as they are adept at handling data that are spatially distributed, including solutions to partial differential equations. When applied to unsteady physics problems, ROMs also require a model for time-series prediction of the low-dimensional latent variables. Long short-term memory (LSTM) networks, a type of recurrent neural network useful for modeling sequential data, are frequently employed in data-driven ROMs for autoregressive time-series prediction. When making predictions at unseen design points over long time horizons, error propagation is a frequently encountered issue, where errors made early on can compound over time and lead to large inaccuracies. In this work, we propose using bagging, a commonly used ensemble learning technique, to develop a fully data-driven ROM framework referred to as the CAE-eLSTM ROM that uses CAEs for spatial reconstruction of the full-order model and LSTM ensembles for time-series prediction. When applied to two unsteady fluid dynamics problems, our results show that the presented framework effectively reduces error propagation and leads to more accurate time-series prediction of latent variables at unseen points.
BibTeX
@misc{halder2024reduced,
  author = {Rakesh Halder and Mohammadmehdi Ataei and Hesam Salehipour and Krzysztof Fidkowski and Kevin Maki},
  title = {{Reduced-order modeling of unsteady fluid flow using neural network ensembles}},
  year = {2024},
  eprint = {2402.05372},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.05372},
}
Figure