Exploring Model Misspecification in Statistical Finite Elements via Shallow Water Equations

Connor Duffin, Paul Branson, Matthew D. Rayson, Mark Girolami, Edward Cripps, Thomas StemlerarXiv 2023 · DOI · arXiv

TL;DR

This paper investigates how the Statistical Finite Element Method (statFEM) handles "severe" model misspecification—situations where the underlying physics equations are fundamentally wrong or use incorrect parameters. Using 1D shallow water equations as a testbed, the authors demonstrate that statFEM is remarkably robust, especially when spatial observation density is high, and can even favor simpler linear models when complex nonlinear parameters are poorly known.

What Problem Does This Paper Address?

In the words of George Box, "all models are wrong, but some are useful." In engineering and geophysics, we often use Partial Differential Equations (PDEs) to describe reality, but these equations are always approximations. We might ignore friction, simplify the geometry, or linearize complex interactions to make the math tractable. This is "model misspecification."

The Statistical Finite Element Method (statFEM) was recently proposed to bridge this gap by embedding Gaussian Process (GP) uncertainty directly into the finite element framework. While statFEM has proven effective for "mild" errors, its performance under "severe" misspecification—where the modeler is significantly wrong about the physics—remained an open question. This paper systematically stress-tests statFEM to see if it can still produce accurate results when the "prior" physics are intentionally broken.

Methods

The authors use the 1D Shallow Water Equations (SWE), which describe the flow of water in an inlet, as their experimental system. Their approach involves several key components:

  • statFEM Framework: They augment the standard PDEs with additive Gaussian Process forcing terms. This allows the model to "admit" it doesn't know everything and adjust its predictions based on incoming data.
  • Discretization: They use a standard Finite Element Method (FEM) with P2-P1 element pairs for velocity and surface height, and a $\theta$-method for time-stepping.
  • Filtering: To update the model with data in real-time, they employ a Low-Rank Extended Kalman Filter (LR-ExKF). The "low-rank" part is crucial for making the Bayesian updates computationally feasible for high-dimensional PDE states.
  • Deliberate Misspecification: They test three types of errors:
  • Parametric: Using the wrong viscosity ($\nu$) or bottom topography (bathymetry, $s$).
  • Structural: Using a linear version of the equations to predict a nonlinear reality.
  • Observational: Varying how often (time) and where (space) data is collected.

Datasets

The study relies on a synthetic dataset.

  • Ground Truth: Generated using a high-fidelity, fully nonlinear SWE simulation with specific "true" parameters ($\nu=1, s=2000$).
  • Test Models: 750 different configurations were run, varying viscosity, shore position, and observation frequencies to see how far they strayed from the ground truth.
  • Code Availability: The authors provided an online repository for reproducibility: https://github.com/connor-duffin/sswe.

Key Insights and Ideas

  • Spatial Data is King: Increasing the number of sensor locations (spatial frequency) reduced model error much more drastically than simply taking measurements more often (temporal frequency). This suggests that for long-wavelength phenomena like tides, seeing the "shape" of the water is more informative than seeing its "pulse."
  • Viscosity vs. Bathymetry: The model was much more sensitive to errors in viscosity than to errors in the shape of the seafloor. High viscosity tends to dominate the dynamics, "washing out" other physical details.
  • The "Linearity" Paradox: Surprisingly, when parameters were severely misspecified, the linearized model actually outperformed the nonlinear one. The authors posit that in highly uncertain regimes, the added complexity of nonlinear interactions just introduces more noise, whereas a linear model provides a cleaner "physics-based interpolator."
  • Uncertainty Quantification (UQ): statFEM successfully increased uncertainty in unobserved variables (like velocity) when the observed variables (surface height) showed a mismatch with the model.

Key Results

  • Accuracy: statFEM performed well across various regimes, with accuracy measured by Root Mean Square Error (RMSE) and Log-Likelihood.
  • Bayes Factors: Using Bayes factors for model comparison, the authors found "strong evidence" (log $BF_{10} \approx 10^5$) favoring well-specified models, but noted that statFEM's ability to correct the mean state remained high even in poor models.
  • Damping Effects: As viscosity increased beyond $10^4$, the distinction between different bathymetry models vanished, as viscous effects began to dominate the flow entirely.

Limitations and Future Work

  • Fixed Hyperparameters: The Gaussian Process hyperparameters (length-scales and variances) were fixed. The authors acknowledge that jointly estimating these parameters alongside the physical state would be a logical next step to improve adaptability.
  • 1D Simplification: While 1D SWE is a standard benchmark, real-world oceanographic problems are 3D and involve much more complex nonlinearities (like turbulence) not captured here.
  • Temporal Correlation: The current GP forcing assumes errors are uncorrelated in time. Incorporating temporal correlation could help the model better handle persistent, systematic physical errors.

Summary updated

Abstract

The abundance of observed data in recent years has increased the number of statistical augmentations to complex models across science and engineering. By augmentation we mean coherent statistical methods that incorporate measurements upon arrival and adjust the model accordingly. However, in this research area methodological developments tend to be central, with important assessments of model fidelity often taking second place. Recently, the statistical finite element method (statFEM) has been posited as a potential solution to the problem of model misspecification when the data are believed to be generated from an underlying partial differential equation system. Bayes nonlinear filtering permits data driven finite element discretised solutions that are updated to give a posterior distribution which quantifies the uncertainty over model solutions. The statFEM has shown great promise in systems subject to mild misspecification but its ability to handle scenarios of severe model misspecification has not yet been presented. In this paper we fill this gap, studying statFEM in the context of shallow water equations chosen for their oceanographic relevance. By deliberately misspecifying the governing equations, via linearisation, viscosity, and bathymetry, we systematically analyse misspecification through studying how the resultant approximate posterior distribution is affected, under additional regimes of decreasing spatiotemporal observational frequency. Results show that statFEM performs well with reasonable accuracy, as measured by theoretically sound proper scoring rules.
BibTeX
@misc{duffin2023exploring,
  author = {Connor Duffin and Paul Branson and Matthew D. Rayson and Mark Girolami and Edward Cripps and Thomas Stemler},
  title = {{Exploring Model Misspecification in Statistical Finite Elements via Shallow Water Equations}},
  year = {2023},
  eprint = {2307.05334},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2307.05334},
}
Figure