Bayesian Data-Driven Discovery of Partial Differential Equations with Variable Coefficients

Aoxue Chen, Yifan Du, Liyao Gao, Guang LinSSRN 2024 · DOI

TL;DR

This paper introduces a novel Bayesian sparse learning algorithm, tBGL-SS, for discovering partial differential equations (PDEs) with spatially or temporally varying coefficients directly from data. It addresses the challenge of noise and model selection by incorporating uncertainty quantification and a new "Bayesian total error bar" criterion, demonstrating superior robustness and accuracy compared to existing methods.

What Problem Does This Paper Address?

Identifying the underlying physical laws governing complex systems is a fundamental challenge in science and engineering. Partial Differential Equations (PDEs) are often used to describe these laws. While data-driven methods have emerged for discovering PDEs with constant coefficients, accurately identifying PDEs with variable coefficients (where parameters change across space or time) has remained a significant hurdle. This difficulty stems from the inherent complexity of distinguishing parameter variations from the system's dynamics, compounded by the presence of noise in real-world data. Furthermore, selecting the correct PDE model from a vast library of potential terms, especially under noisy conditions, is a non-trivial task.

Methods

The core of this work is the proposed threshold Bayesian group Lasso with spike-and-slab prior (tBGL-SS) algorithm. This method builds upon Bayesian group Lasso, which is designed to handle grouped sparsity (where coefficients within a group are either all zero or all non-zero), a suitable structure for variable coefficients. The "spike-and-slab prior" is incorporated to encourage sparsity, effectively selecting relevant terms for the PDE.

What makes tBGL-SS novel and efficient is the thresholding procedure. Instead of relying on computationally expensive full MCMC sampling for every iteration, tBGL-SS uses an approximate MCMC approach. It incorporates a hard-thresholding step to remove less likely terms from the candidate library, significantly reducing computational burden without sacrificing accuracy. This thresholding is guided by the posterior distribution of the coefficients.

For model selection, the paper introduces a Bayesian total error bar criterion. This metric leverages the uncertainty estimates from the Bayesian framework to select the most appropriate model, outperforming traditional metrics like Mean Squared Error (MSE) and Akaike Information Criterion (AIC)-inspired losses, especially in noisy scenarios.

Datasets

The paper evaluates the tBGL-SS algorithm on four benchmark PDE problems:

  • Linear advection equation: Used to demonstrate the identification of a spatially dependent wave speed.
  • Viscous Burger's equation: Features a time-dependent oscillating coefficient.
  • Advection-diffusion equation: Tests the identification of a spatially varying coefficient.
  • Kuramoto-Sivashinsky equation: A complex system known for chaotic behavior, used to test robustness with higher-order derivatives.

The data for these problems were generated from reference numerical simulations with pre-determined initial and boundary conditions. The authors also injected varying levels of white noise into these datasets to assess the robustness of their method.

Key Insights and Ideas

  • Thresholding for Efficiency: The integration of a thresholding step into the Bayesian group Lasso framework is a key innovation that makes the method computationally tractable for complex PDEs with many potential terms, especially those with variable coefficients.
  • Uncertainty as a Guide: The paper strongly advocates for using uncertainty quantification not just for estimating coefficients but as a crucial component for model selection. This is a departure from methods that solely rely on point estimates.
  • Bayesian Total Error Bar Superiority: The proposed Bayesian total error bar criterion proves to be a more reliable metric for model selection than traditional methods, particularly when dealing with noisy data or when distinguishing between models with subtle differences. It effectively penalizes models that have large unexplained variance.
  • Robustness to Noise: tBGL-SS demonstrates a significant improvement in robustness to noise compared to existing methods like SGTR and Group Lasso, a critical factor for real-world applications.

Key Results

The tBGL-SS algorithm consistently outperformed baseline methods (SGTR and Group Lasso) in identifying PDEs with variable coefficients, especially under noisy conditions.

  • In the linear advection equation example, tBGL-SS correctly identified the first-order derivative term and the spatially dependent wave speed, while Group Lasso falsely identified a diffusion term.
  • For Burger's equation, tBGL-SS showed more robust coefficient identification under increasing noise levels compared to SGTR and Group Lasso.
  • In the advection-diffusion equation, tBGL-SS again demonstrated better performance with larger noise, correctly identifying terms that Group Lasso and SGTR missed.
  • For the Kuramoto-Sivashinsky equation, while all methods struggled with high noise due to the fourth-order derivative, tBGL-SS showed better performance with low noise.
  • The Bayesian total error bar criterion consistently selected the correct sparsity pattern, whereas MSE favored full models and AIC-like loss struggled to distinguish between models with minor term differences.

Limitations and Future Work

  • Threshold Tuning: The paper acknowledges that the value of thresholds in the tBGL-SS algorithm needs careful tuning. If set too low, it can lead to overfitting; if set too high, it may exclude correct terms.
  • Computational Cost for Very High Dimensions: While significantly more efficient than standard Bayesian methods, the computational cost can still be substantial for extremely large numbers of variables.
  • Future Work: The authors suggest exploring different preprocessing methods for noise filtering to further improve results, especially in the presence of very large observation noise. They also implicitly suggest that further investigation into optimal threshold selection strategies could be beneficial.

Summary updated

BibTeX
@misc{chen2024bayesian,
  author = {Aoxue Chen and Yifan Du and Liyao Gao and Guang Lin},
  title = {{Bayesian Data-Driven Discovery of Partial Differential Equations with Variable Coefficients}},
  year = {2024},
  doi = {10.2139/ssrn.4747393},
  howpublished = {SSRN},
}
Figure