Statistical finite elements for misspecified models

Connor Duffin, Edward Cripps, Thomas Stemler, Mark GirolamiProceedings of the National Academy of Sciences 2020 · DOI

TL;DR

This paper introduces a novel statistical finite element method (statFEM) that integrates data assimilation into nonlinear partial differential equation (PDE) models, specifically addressing the challenge of model misspecification. By treating model discrepancies as stochastic forcing within a Bayesian framework and employing Kalman filtering, statFEM can accurately reconstruct phenomena even when the underlying physical model is imperfect.

What Problem Does This Paper Address?

Many real-world phenomena in science and engineering are described by complex, nonlinear, time-dependent PDEs. While the finite element method (FEM) is a powerful tool for numerically solving these PDEs, empirical data often reveals discrepancies between model predictions and reality. This "model misspecification" is a significant challenge, as it can lead to inaccurate predictions and a lack of confidence in the model's output. Traditional methods often struggle to reconcile these differences, either by trying to perfectly tune model parameters or by ignoring the data. This paper tackles this gap by proposing a framework that explicitly accounts for model misspecification and uses data to update the FEM solution in a statistically rigorous way.

Methods

The core of the proposed method, statFEM, is a Bayesian approach that embeds a Gaussian process (GP) as stochastic forcing within the governing PDE. This GP represents unknown processes or model errors. The FEM is then used to discretize the PDE, transforming the problem into a nonlinear Gaussian state-space model.

The key innovation lies in how they handle the resulting complex probability distributions. Since a closed-form solution for the posterior distribution is often intractable for nonlinear PDEs, they employ two data assimilation techniques:

  • Extended Kalman Filter (EKF): This method linearizes the nonlinear operators around the current solution to approximate the posterior distribution as Gaussian.
  • Ensemble Kalman Filter (EnKF): This approach uses an ensemble of model states to represent the uncertainty and computes summary statistics (mean and covariance) from the ensemble. The paper notes that the EnKF is generally more robust, especially when dealing with high-gradient regions or large time steps, as it avoids the covariance inflation issues sometimes seen with the EKF.

The method sequentially updates the FEM solution by incorporating observed data through a filtering framework. This involves a prediction step (propagating the model forward in time) and an analysis step (updating the state based on new observations). Crucially, the method also includes a mechanism for estimating hyperparameters of the stochastic forcing and observation noise, often using an empirical Bayes approach by maximizing the log-marginal posterior.

Datasets

The paper uses two main sources of data for evaluation:

  1. Simulated Data: They generate data from an extended Korteweg-de Vries (eKdV) equation with a cubic nonlinear term. This simulated data is then "jittered" with synthetic Gaussian observational noise to mimic real-world measurement imperfections.
  2. Experimental Data: They apply their method to experimental data from a study on weakly nonlinear internal waves in lakes, as reported in a previous publication. This data was collected using ultrasonic wave gauges in a controlled laboratory setting.

Key Insights and Ideas

  • Model Misspecification as Stochastic Forcing: Explicitly modeling unknown or imperfect aspects of a physical model as a Gaussian process is a powerful way to quantify uncertainty arising from model limitations.
  • Data Assimilation for FEM: Integrating Kalman filtering techniques (EKF and EnKF) with FEM provides a principled way to update numerical solutions with sparse or noisy data, leading to more accurate and reliable predictions.
  • Interpretable Uncertainty Quantification: The resulting posterior distribution provides a statistically coherent measure of uncertainty, allowing researchers to understand the confidence in their model's predictions and identify areas where the model is most uncertain.
  • Reconstruction of Complex Phenomena: The method demonstrates a remarkable ability to reconstruct complex phenomena, such as the interaction of solitons in the KdV equation and the wave profiles of internal waves, even when the initial model is a poor fit.

Key Results

  • Improved Model Accuracy: In both simulation and experimental studies, the statFEM approach significantly reduced model discrepancy compared to the deterministic FEM solution. For instance, in the simulation study (Fig. 1), the posterior mean closely matched the data-generating process, correcting for phase shifts and oscillations present in the prior.
  • Accurate Reconstruction of Wave Phenomena: The method successfully reconstructed the observed wave profiles in the experimental data, providing a close fit to measurements at wave gauge locations and capturing the underlying wave dynamics.
  • Shrinking Credible Intervals: Conditioning on data led to a substantial reduction in the uncertainty bounds (95% credible intervals) around the posterior mean, indicating increased confidence in the model's updated predictions.
  • Parameter Estimation: The method effectively estimated hyperparameters of the stochastic forcing and observation noise, providing insights into the model's accuracy over time. For example, the scale parameter in the simulation study indicated varying accuracy of model predictions.

Limitations and Future Work

  • Computational Cost: While the paper presents two algorithms, Kalman filtering methods, especially the EnKF with large ensembles, can be computationally intensive for very large-scale problems.
  • Assumptions of Gaussianity: The EKF and EnKF inherently rely on Gaussian assumptions for the posterior distribution. For highly nonlinear systems with strongly non-Gaussian uncertainties, these approximations might become less accurate.
  • Time-Invariant Parameters: The paper mentions that time-invariant parameters can be handled, but the primary focus is on sequential updating. Further exploration of robust methods for estimating time-varying parameters would be valuable.
  • Future Directions: The authors suggest several avenues for future research:
  • Exploring alternative discretization schemes (e.g., Crank-Nicolson for stability).
  • Investigating more advanced data assimilation techniques for high-dimensional problems.
  • Extending the methodology to other discretizations, such as spectral methods.
  • Developing methods for uncertainty quantification of physical parameters themselves, beyond the stochastic forcing.

Summary updated

Abstract

We present a statistical finite element method for nonlinear, time-dependent phenomena, illustrated in the context of nonlinear internal waves (solitons). We take a Bayesian approach and leverage the finite element method to cast the statistical problem as a nonlinear Gaussian state-space model, updating the solution, in receipt of data, in a filtering framework. The method is applicable to problems across science and engineering for which finite element methods are appropriate. The Korteweg-de Vries equation for solitons is presented because it reflects the necessary complexity while being suitably familiar and succinct for pedagogical purposes. We present two algorithms to implement this method, based on the extended and ensemble Kalman filters, and demonstrate effectiveness with a simulation study and a case study with experimental data. The generality of our approach is demonstrated in <i>SI Appendix</i>, where we present examples from additional nonlinear, time-dependent partial differential equations (Burgers equation, Kuramoto-Sivashinsky equation).
BibTeX
@inproceedings{duffin2020statistical,
  author = {Connor Duffin and Edward Cripps and Thomas Stemler and Mark Girolami},
  title = {{Statistical finite elements for misspecified models}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2020},
  doi = {10.1073/pnas.2015006118},
}

Loading related content...

Figure