Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges

Karan Jakhar, Yifei Guan, Rambod Mojgani, Ashesh Chattopadhyay, Pedram Hassanzadeh, Laura ZannaJournal of Advances in Modeling Earth Systems 2023 · DOI · OpenReview

TL;DR

This paper explores using equation discovery techniques to learn interpretable subgrid-scale (SGS) closures for fluid dynamics from high-fidelity simulation data. While the discovered closures are highly accurate in predicting SGS fluxes (correlation > 0.95), they lead to unstable simulations, highlighting a critical gap between structural accuracy and functional stability in SGS modeling.

What Problem Does This Paper Address?

Representing the effects of small-scale turbulent processes (subgrid-scale or SGS) is crucial for simulating complex systems like Earth's atmosphere and oceans. These processes are too fine to be directly resolved by computational models, so they are approximated by "closures" or "parameterizations." Traditional physics-based parameterizations often have shortcomings, leading to biases. Machine learning, particularly equation discovery, offers a promising avenue for learning interpretable, closed-form SGS closures directly from data. However, previous attempts have yielded closures that are highly accurate in matching observed SGS fluxes but lead to unstable simulations when integrated into larger models. This paper aims to understand why this disconnect occurs and propose ways to develop accurate and stable closures.

Methods

The authors employ Relevance Vector Machine (RVM), a sparsity-promoting Bayesian regression technique, to discover closed-form equations for SGS closures. They use an expansive library of basis functions, including derivatives of velocity and temperature up to 8th order and their quadratic combinations. This library is designed to capture complex nonlinear relationships. The RVM is trained to minimize the mean squared error (MSE) between the discovered closure and the "truth" SGS fluxes derived from filtered direct numerical simulations (FDNS). A key aspect is the use of different filtering procedures (Gaussian, box, Gaussian+box, sharp-spectral) and filter sizes to assess the robustness of the discovered closures and understand their sensitivity to data processing.

Datasets

The study utilizes filtered direct numerical simulation (FDNS) data from two canonical test cases:

  • 2D forced homogeneous isotropic turbulence (2D-FHIT)
  • Turbulent Rayleigh-Bénard convection (RBC)

These FDNS datasets are treated as the "ground truth" for training and evaluating the discovered SGS closures. The paper does not introduce new benchmark datasets but rather uses established simulation setups to generate data for analysis.

Key Insights and Ideas

  • NGM Dominance: Equation discovery algorithms, when provided with an expansive library and a standard MSE loss function, robustly discover the Nonlinear Gradient Model (NGM). This is because the NGM represents the leading term in the Taylor-series expansion of the SGS flux, which dominates the MSE.
  • Structural vs. Functional Accuracy: The discovered NGM closures exhibit excellent a priori (offline) accuracy (correlation coefficients often > 0.95), meaning they accurately predict SGS fluxes. However, this structural accuracy does not translate to a posteriori (online) stability when integrated into LES simulations.
  • Filter Dependence: The discovered closures are sensitive to the choice of filter type and size. Specifically, sharp-spectral (cutoff) filters lead to poor discovery results, likely because they are not reversible and information is lost.
  • Inter-scale Energy Transfer is Key: In 2D turbulence, the NGM's inability to capture any inter-scale kinetic energy transfer (both forward and backward) is identified as a primary reason for LES instability.
  • Backscatter Matters: In RBC, while energy transfer is not zero, the poor prediction of potential energy backscatter is identified as a potential cause of instability.
  • "Truth" is Not Unique: The diagnosed SGS fluxes from FDNS data are not unique and depend on the filtering procedure, highlighting a critical challenge for data-driven modeling.

Key Results

  • Robust Discovery of NGM: For both 2D-FHIT and RBC, and across various filter types and sizes (except sharp-spectral), the RVM consistently discovers SGS closures that are structurally identical to the analytically derived Nonlinear Gradient Model (NGM).
  • High A Priori Accuracy: The discovered NGM closures achieve very high correlation coefficients (CC) with the true SGS fluxes, often exceeding 0.95 and reaching up to 0.99.
  • Universal Instability: Despite high a priori accuracy, all tested LES simulations using the discovered NGM closures become unstable, even at high resolutions. This is a consistent finding across all tested cases.
  • Filter Impact: Sharp-spectral filters lead to low CC and non-robust results, indicating a failure of the equation discovery process for this filter type.
  • Component Importance: The analysis of Leonard, cross, and Reynolds stresses reveals that the Leonard and cross stress components often dominate the total SGS flux, especially for filters with compact support. This suggests that parameterizing only the Reynolds stress might be insufficient.

Limitations and Future Work

  • Instability of NGM: The primary limitation is the observed instability of LES simulations using the discovered NGM closures, despite their high a priori accuracy. This suggests that matching SGS flux statistics is insufficient for stable LES.
  • Filter Dependence of "Truth": The fact that SGS fluxes diagnosed from FDNS are not unique and depend on the filtering procedure poses a challenge for defining the "true" SGS flux for training.
  • MSE Loss Function Bias: The standard MSE loss function inherently favors the leading terms of the Taylor-series expansion, leading to the discovery of NGM. This limits the ability to discover closures that might capture other important physics not represented by the first term.
  • Future Work Suggestions:
    • Physics-Informed Loss Functions: Incorporate functional modeling metrics (e.g., inter-scale energy transfer, backscatter) directly into the loss function to guide the discovery towards more physically consistent closures.
  • Flexible Libraries and Methods: Explore methods like genetic programming or gene expression programming that can evolve libraries, or use deep neural networks with flexible loss functions.
  • Investigate "Truth" Definition: Further examine how the choice of filter and data processing affects the diagnosed SGS fluxes and how this impacts closure discovery.
  • Functional Modeling Approaches: Explore alternative equation-discovery approaches that are more directly focused on functional modeling, such as learning from differences between high- and low-resolution simulations.

Summary updated

Abstract

There is growing interest in discovering interpretable, closed-form equations for subgrid-scale (SGS) closures/parameterizations of complex processes in Earth systems. Here, we apply a common equation-discovery technique with expansive libraries to learn closures from filtered direct numerical simulations of 2D turbulence and Rayleigh-B\'enard convection (RBC). Across common filters (e.g., Gaussian, box), we robustly discover closures of the same form for momentum and heat fluxes. These closures depend on nonlinear combinations of gradients of filtered variables, with constants that are independent of the fluid/flow properties and only depend on filter type/size. We show that these closures are the nonlinear gradient model (NGM), which is derivable analytically using Taylor-series. Indeed, we suggest that with common (physics-free) equation-discovery algorithms, for many common systems/physics, discovered closures are consistent with the leading term of the Taylor-series (except when cutoff filters are used). Like previous studies, we find that large-eddy simulations with NGM closures are unstable, despite significant similarities between the true and NGM-predicted fluxes (correlations $> 0.95$). We identify two shortcomings as reasons for these instabilities: in 2D, NGM produces zero kinetic energy transfer between resolved and subgrid scales, lacking both diffusion and backscattering. In RBC, potential energy backscattering is poorly predicted. Moreover, we show that SGS fluxes diagnosed from data, presumed the ''truth'' for discovery, depend on filtering procedures and are not unique. Accordingly, to learn accurate, stable closures in future work, we propose several ideas around using physics-informed libraries, loss functions, and metrics. These findings are relevant to closure modeling of any multi-scale system.
BibTeX
@article{jakhar2023learning,
  author = {Karan Jakhar and Yifei Guan and Rambod Mojgani and Ashesh Chattopadhyay and Pedram Hassanzadeh and Laura Zanna},
  title = {{Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges}},
  journal = {Journal of Advances in Modeling Earth Systems},
  year = {2023},
  doi = {10.1029/2023ms003874},
}
Figure