DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks

Shengze Cai, Zhicheng Wang, Lu Lu, Tamer A. Zaki, George Em KarniadakisJournal of Computational Physics 2021 · OpenReview

TL;DR

This paper introduces DeepM&Mnet, a novel deep learning framework that leverages pre-trained DeepONets to efficiently infer complex multiphysics fields from sparse measurements. By combining operator approximation with data assimilation, DeepM&Mnet achieves high accuracy and significant speedups compared to traditional numerical methods for problems like electroconvection.

What Problem Does This Paper Address?

Simulating multiphysics and multiscale problems, such as electroconvection, is computationally expensive. Electroconvection involves the intricate coupling of fluid flow, electric fields, and ion concentrations, leading to steep boundary layers that are challenging for standard numerical methods. While these methods can be accurate, they require substantial computational resources and time. Furthermore, real-world applications often involve inferring these complex fields from limited, sparse measurements, which is a difficult data assimilation problem for conventional approaches. The paper aims to bridge this gap by developing a data-driven framework that can accurately and efficiently predict these fields using neural networks, even with minimal data.

Methods

The core of the proposed solution lies in two key components: DeepONets and the DeepM&Mnet framework.

  • DeepONets (Deep Operator Networks): These are neural networks designed to learn nonlinear operators. A DeepONet consists of two sub-networks: a "branch net" that takes an input function and a "trunk net" that takes spatial coordinates. By combining their outputs, DeepONets can approximate mappings from functions to functions, making them suitable for learning differential equations or relationships between physical fields. In this work, individual DeepONets were trained to predict each field (velocity, potential, concentrations) independently, given inputs from the other fields.
  • DeepM&Mnet (Deep Multiphysics & Multiscale Network): This is a novel framework that integrates pre-trained DeepONets with a neural network surrogate model. The DeepONets act as fixed "building blocks" or constraints, enforcing the physics-based relationships between the fields. A separate, fully-connected neural network is trained to approximate the overall multiphysics solution. This network takes spatial coordinates as input and outputs all the coupled fields. The training process minimizes a combined loss function that includes data mismatch (from sparse measurements) and operator loss (ensuring consistency with the pre-trained DeepONets). Two architectures are proposed:
  • Parallel DeepM&Mnet: A single neural network predicts all fields, with DeepONets providing constraints.
  • Series DeepM&Mnet: A neural network predicts one primary field (e.g., electric potential), and DeepONets are used sequentially to derive the other coupled fields.

Datasets

The primary dataset used in this paper is generated through high-fidelity numerical simulations of the 2D electroconvection problem using a spectral element solver (NekTar).

  • Training Data: The authors generated 15 steady-state snapshots of the electroconvection problem by varying the applied electric potential difference (ΔΦ) across a range of values. For each snapshot, the fields of electric potential ($\phi$), velocity ($u, v$), and ion concentrations ($c^+, c^-$) were collected. These datasets were used to train individual DeepONets.
  • Testing Data: Additional simulations were performed for unseen electric potential differences (ΔΦ = 13.4 and ΔΦ = 62.15) to evaluate the performance of both individual DeepONets and the DeepM&Mnet framework.
  • Benchmark: The electroconvection problem itself serves as a benchmark for multiphysics and multiscale modeling. The authors note that the physical parameters used in their simulations (e.g., Debye length) are chosen to represent realistic scenarios, and the problem exhibits multiscale behavior due to steep boundary layers.

Key Insights and Ideas

  • DeepONets as Physics Building Blocks: The paper demonstrates that DeepONets can effectively learn the complex, nonlinear relationships between coupled physical fields in a multiphysics system, acting as powerful, pre-trained physics modules.
  • Data Assimilation with Operator Constraints: DeepM&Mnet offers a flexible and efficient way to perform data assimilation. By combining sparse measurements with the physics encoded in pre-trained DeepONets, it can infer full fields even when measurements are scarce and not available for all variables.
  • "Plug-and-Play" Modularity: The pre-trained DeepONets can be treated as fixed components, allowing for a "plug-and-play" approach to build complex multiphysics models. This modularity simplifies the development and adaptation of the framework.
  • Handling Multiscale Phenomena: The use of MAPE loss for velocity prediction is crucial for accurately capturing multiscale velocity fields, where magnitudes can vary drastically. This highlights the importance of choosing appropriate loss functions for different physical quantities.
  • Efficiency Gains: The DeepM&Mnet framework achieves significant speedups (up to 10,000x) compared to traditional numerical solvers, making it practical for rapid prediction and simulation.

Key Results

  • DeepONet Accuracy: Individual DeepONets were trained to predict fields with high accuracy. For example, MSEs for testing DeepONets were on the order of $10^{-6}$ to $10^{-5}$, with relative L2-norm errors typically below 1%.
  • DeepM&Mnet Performance: The parallel DeepM&Mnet achieved excellent results for the electroconvection problem. With only 20 measurements per field, it could infer full fields with L2-norm errors generally below 4% for all variables.
  • Sensor Efficiency: The DeepM&Mnet framework demonstrated remarkable efficiency in terms of data requirements. It achieved good accuracy (less than 10% error) with as few as 10 measurements per field, whereas a standard neural network without operator constraints required hundreds of sensors for comparable accuracy.
  • Speedup: The prediction speed of trained DeepONets was found to be less than 1 second on a common GPU, offering a speedup of about 10,000-fold compared to the NekTar simulation for independent conditions.
  • Series Architecture Effectiveness: The series DeepM&Mnet also showed promising results, achieving relative L2-norm errors for the primary field ($\phi$) around 0.76% with 20 measurements.

Limitations and Future Work

  • Pre-training Requirement: The DeepM&Mnet framework relies on pre-trained DeepONets. The initial training of these DeepONets requires labeled data, which can still be computationally intensive. However, this training is performed offline, and the DeepONets are then used in a fast, "plug-and-play" manner.
  • Generalizability: While the paper suggests the framework is general and can be applied to other multiphysics and multiscale problems (e.g., hypersonics with finite chemistry), further validation on diverse problem domains is needed.
  • Future Work: The authors explicitly mention that future work will involve applying this framework to other complex problems, such as hypersonics with finite chemistry. They also suggest that including pressure fields in the DeepONet and DeepM&Mnet architectures would be straightforward. The influence of L2 regularization was investigated in an appendix, suggesting further exploration of regularization techniques.

Summary updated

Abstract

Highlights • DeepONets can learn the electroconvection operator using a small number of data. • DeepONets achieve high accuracy at 1000X speed for the unseen testing data. • DeepM&Mnet is proposed as a new data assimilation framework. • DeepM&Mnet enables the combination of data measurements and pre-trained DeepONets. • A comparison between two DeepM&Mnet architectures is performed. Abstract Electroconvection is a multiphysics problem involving coupling of the flow field with the electric field as well as the cation and anion concentration fields. Here, we use electroconvection as a benchmark problem to put forward a new data assimilation framework, the DeepM&Mnet, for simulating multiphysics and multiscale problems at speeds much faster than standard numerical methods using pre-trained neural networks. We first pre-train DeepONets that can predict independently each field, given general inputs from the rest of the fields of the coupled system. DeepONets can approximate nonlinear operators and are composed of two sub-networks, a branch net for the input fields and a trunk net for the locations of the output field. DeepONets, which are extremely fast, are used as building blocks in the DeepM&Mnet and form constraints for the multiphysics solution along with some sparse available measurements of any of the fields. We demonstrate the new methodology and document the accuracy of each individual DeepONet, and subsequently we present two different DeepM&Mnet architectures that infer accurately and efficiently 2D electroconvection fields for unseen electric potentials. The DeepM&Mnet framework is general and can be applied for building any complex multiphysics and multiscale models based on very few measurements using pre-trained DeepONets in a “plug-and-play” mode.
BibTeX
@article{cai2021deepm,
  author = {Shengze Cai and Zhicheng Wang and Lu Lu and Tamer A. Zaki and George Em Karniadakis},
  title = {{DeepM\&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks}},
  journal = {Journal of Computational Physics},
  year = {2021},
  url = {https://openreview.net/forum?id=rCgb7GSVf75},
}
Figure