Bayesian Nonparametric Generative Modeling of Large Multivariate Non-Gaussian Spatial Fields

Paul F. V. Wiemann, Matthias KatzfußJournal of Agricultural Biological and Environmental Statistics 2023 · DOI · arXiv

TL;DR

This paper introduces a scalable Bayesian framework for modeling large, multivariate spatial fields that don't follow a normal (Gaussian) distribution. By combining "triangular transport maps" with a clever augmented input space, the authors create a generative model that captures complex inter-variable dependencies and non-stationarity, even when only a handful of training samples are available.

What Problem Does This Paper Address?

In fields like climate science, researchers deal with "multivariate spatial fields"—multiple related variables (like temperature, precipitation, and soil moisture) spread across a geographic area. Modeling these is hard for three reasons:

  1. Non-Gaussianity: Real-world data rarely follows a perfect bell curve; it can be skewed or have "heavy tails."
  2. Non-stationarity: The relationship between variables often changes depending on where you are (e.g., the link between humidity and rain is different in a desert vs. a rainforest).
  3. Data Scarcity: High-fidelity climate models are incredibly expensive to run. While deep learning models like GANs or VAEs are great at generating data, they typically require thousands of samples. Climate scientists might only have 40 or 50.

Existing statistical methods usually assume the data is Gaussian or struggle to scale to the massive number of grid points found in modern climate outputs.

Methods

The authors build upon the Bayesian Transport Map (BTM). A transport map is a mathematical function that transforms a complex, "messy" distribution into a simple, standard Gaussian distribution.

The "secret sauce" in this paper is the Augmented Input Space. Instead of just looking at spatial coordinates ($s$), they add a latent "process space" ($\breve{s}$). Imagine stacking different variables (like rain and temperature) into a higher-dimensional space. If two variables are highly correlated, the model places them "close" to each other in this latent space.

Key technical highlights: * Gaussian Processes (GPs): The components of the transport map are modeled as GPs, allowing for a nonparametric fit that doesn't force the data into a specific functional shape. * Vecchia Approximation: To handle "large" data (thousands of locations), they use a sparse approximation that only considers a small set of "neighboring" points for each calculation, bringing the computational complexity down to a manageable level. * Empirical Bayes: They use stochastic gradient descent (the Adam optimizer) to find the best hyperparameters, making the inference process highly scalable and compatible with modern machine-learning hardware.

Datasets

The model was evaluated using two primary sources:

  1. Simulation Study: A synthetic 5-variate spatial field generated on a $32 \times 32$ grid. This allowed the authors to test the model against a "ground truth" where they knew exactly how much non-linearity was present.
  2. Climate Model Output: Data from the Community Earth System Model (CESM) Large Ensemble (LENS) project. They focused on four hydrological variables over the United States: Precipitation (PRECIP), Soil Water (SOILWATER_10CM), and Runoff (QRUNOFF). The dataset consisted of 42 ensemble members (replicates).

Key Insights and Ideas

  • Univariate to Multivariate via Geometry: The most elegant idea here is that by adding a latent dimension for "variable type," you can use a univariate spatial model to solve a multivariate problem. You treat the different variables simply as different locations in a larger, augmented space.
  • Efficiency with Small $R$: Unlike GANs, which might need 10,000 images to learn what a face looks like, this model starts outperforming standard parametric models with as few as 20 to 25 replicates. This is a game-changer for expensive scientific simulations.
  • Capturing Anisotropy: Standard models often produce "too smooth" results. The authors found that their Multivariate Transport Map (MVTM) was much better at capturing the directional patterns (anisotropy) and "sharpness" present in real climate data.

Key Results

  • Higher Log-Density: In both simulations and real climate data, the MVTM consistently achieved higher log-density (a measure of model fit) on holdout test data compared to standard parametric Gaussian Process models.
  • Superior Conditional Modeling: The model excelled at predicting one variable given others (e.g., predicting Runoff if you already know the Rainfall and Soil Moisture). This is vital for "emulation," where a statistical model stands in for a heavy physics simulation.
  • Speed: The training is remarkably fast. On a standard laptop (Apple M1 Pro), the entire estimation process for the climate dataset took between 1.5 to 3 minutes, depending on the number of replicates used.

Limitations and Future Work

  • The "Small $P$" Constraint: The current parameterization requires $P(P-1)/2$ entries for the process positions. While this works great for 5 or 10 variables, it becomes a bottleneck if you want to model all 1,168 variables produced by the CESM.
  • Conditional Normality: While the joint distribution is non-Gaussian, the model still assumes conditional normality at each location. The authors suggest relaxing this in the future to better handle extreme values (like 1-in-100-year floods).
  • Time and Covariates: The current version is purely spatial. The authors aim to add a temporal component (to capture how these fields evolve over years) and include external covariates (like elevation or land use) to improve accuracy.

Summary updated

BibTeX
@article{wiemann2023bayesian,
  author = {Paul F. V. Wiemann and Matthias Katzfuß},
  title = {{Bayesian Nonparametric Generative Modeling of Large Multivariate Non-Gaussian Spatial Fields}},
  journal = {Journal of Agricultural Biological and Environmental Statistics},
  year = {2023},
  doi = {10.1007/s13253-023-00580-z},
}
Figure