Weak form theory-guided neural network (TgNN-wf) for deep learning of subsurface single- and two-phase flow

Rui Xu, Dongxiao Zhang, Miao Rong, Nanzhe Wang

TL;DR

This paper introduces a novel "Weak Form Theory-guided Neural Network" (TgNN-wf) that leverages the weak formulation of physical laws to improve the accuracy and robustness of deep learning models for subsurface flow problems. By integrating physics directly into the neural network's loss function, TgNN-wf significantly outperforms traditional physics-informed neural networks, especially in scenarios with discontinuities or noisy data.

What Problem Does This Paper Address?

Subsurface flow problems, such as groundwater movement or hydrocarbon recovery, are governed by complex partial differential equations (PDEs). While deep neural networks (DNNs) show promise as surrogate models for these simulations, their accuracy can degrade when dealing with PDEs that have high-order derivatives or when the solutions exhibit sharp discontinuities (e.g., shock fronts). Existing "theory-guided" DNNs often rely on the "strong form" of PDEs, which can be sensitive to these challenges. This paper addresses the need for a more robust and accurate deep learning approach that can effectively handle these complexities in subsurface flow modeling, particularly in data-scarce environments.

Methods

The core innovation is the development of a Weak Form Theory-guided Neural Network (TgNN-wf). Instead of directly enforcing the strong form of the PDE, TgNN-wf incorporates the weak form formulation into the neural network's loss function. This is achieved by:

  • Integration-by-Parts: High-order derivatives in the PDE are transferred to "test functions" through integration by parts. This reduces the computational burden and error associated with directly computing high-order derivatives.
  • Domain Decomposition: The computational domain is divided into smaller subdomains. This allows for the use of locally defined test functions, which are crucial for capturing local discontinuities and reducing computational cost.
  • Lagrangian Duality: The weights of different loss function components (data mismatch, weak form residual, initial/boundary conditions) are treated as hyperparameters and optimized using a Lagrangian duality approach. This automates the tuning process, which is often done manually in previous methods.

Datasets

The paper uses synthetic data generated from numerical simulations for its case studies. Specifically:

  • 2D Unsteady-State Single-Phase Flow: Data was generated using MODFLOW software for hydraulic head prediction. The first 18 timesteps were used for training, and the remaining 32 for testing.
  • 1D Unsteady-State Buckley-Leverett Two-Phase Flow: Data was generated from analytical solutions of the Buckley-Leverett equation for saturation profile prediction. Training data was obtained from the first 20 timesteps, and the remaining 30 were used for prediction.

No new benchmark datasets were introduced; rather, existing problem formulations were used to evaluate the proposed method.

Key Insights and Ideas

  • Weak Form is Key for Discontinuities: The weak form formulation inherently smooths out noise and is more robust to discontinuities than the strong form, making it ideal for problems with sharp fronts.
  • Integration-by-Parts Simplifies Derivatives: Shifting derivative computations to test functions significantly reduces computational error and cost, especially for high-order PDEs.
  • Domain Decomposition Enhances Local Accuracy: Breaking down the problem allows for localized treatment of discontinuities and more efficient integration.
  • Lagrangian Duality Automates Weight Tuning: Optimizing loss function weights automatically removes a significant manual tuning step, improving reproducibility and efficiency.
  • Robustness to Noisy Data: TgNN-wf demonstrates superior performance even with significant noise in the training data, a critical advantage for real-world applications where data is often imperfect.

Key Results

  • Improved Accuracy: TgNN-wf consistently achieved higher accuracy (lower L2 error, higher R²) compared to the strong-form TgNN and standard DNNs, particularly in the Buckley-Leverett problem with sharp discontinuities.
  • Robustness to Noise: TgNN-wf showed minimal degradation in accuracy with up to 80% noise in the training data, while TgNN and DNN performance suffered significantly.
  • Computational Efficiency: TgNN-wf was generally faster than TgNN, especially when fewer collocation regions were used, due to the efficient handling of derivatives.
  • Small Data Regime Performance: TgNN-wf maintained high prediction accuracy even with limited training data, outperforming TgNN and DNN in these scenarios.
  • Shock Capturing: In the Buckley-Leverett problem, TgNN-wf accurately captured the shock location and maintained its sharpness, a feat that was challenging for other methods.

Limitations and Future Work

  • Test Function Choice: While the paper uses polynomials as test functions, it acknowledges that different forms (e.g., sine functions, Gauss-Legendre polynomials) could be explored for further improvements.
  • Lagrangian Duality Extension: The current work focused on optimizing the weak form residual weight. Future work could involve a more rigorous study to optimize the weights of all loss function terms.
  • Generalization to Other Problems: While successful for subsurface flow, further validation on a wider range of complex, high-dimensional, or discontinuous problems would be beneficial.
  • Overshooting/Undershooting: Despite improvements, some overshooting or undershooting behavior near shocks was still observed in TgNN-wf, suggesting room for further refinement.

Summary updated

Abstract

Deep neural networks (DNNs) are widely used as surrogate models in geophysical applications; incorporating theoretical guidance into DNNs has improved the generalizability. However, most of such approaches define the loss function based on the strong form of conservation laws (via partial differential equations, PDEs), which is subject to deteriorated accuracy when the PDE has high order derivatives or the solution has strong discontinuities. Herein, we propose a weak form theory-guided neural network (TgNN-wf), which incorporates the weak form formulation of the PDE into the loss function combined with data constraint and initial and boundary conditions regularizations to tackle the aforementioned difficulties. In the weak form, high order derivatives in the PDE can be transferred to the test functions by performing integration-by-parts, which reduces computational error. We use domain decomposition with locally defined test functions, which captures local discontinuity effectively. Two numerical cases demonstrate the superiority of the proposed TgNN-wf over the strong form TgNN, including the hydraulic head prediction for unsteady-state 2D single-phase flow problems and the saturation profile prediction for 1D two-phase flow problems. Results show that TgNN-wf consistently has higher accuracy than TgNN, especially when strong discontinuity in the solution is present. TgNN-wf also trains faster than TgNN when the number of integration subdomains is not too large (<10,000). Moreover, TgNN-wf is more robust to noises. Thus, the proposed TgNN-wf paves the way for which a variety of deep learning problems in the small data regime can be solved more accurately and efficiently.
BibTeX
@article{xu2021weak,
  author = {Rui Xu and Dongxiao Zhang and Miao Rong and Nanzhe Wang},
  title = {{Weak form theory-guided neural network (TgNN-wf) for deep learning of subsurface single- and two-phase flow}},
  journal = {Journal of Computational Physics},
  year = {2021},
  doi = {10.1016/j.jcp.2021.110318},
}
Figure