Scalable Bayesian transport maps for high-dimensional non-Gaussian spatial fields

Matthias Katzfuß, Florian Schäfer

TL;DR

This paper introduces a scalable Bayesian framework for learning transport maps—mathematical transformations that turn complex, non-Gaussian spatial data into simple Gaussian noise. By using Gaussian Process (GP) priors to model these maps, the authors create a method that is highly flexible, quantifies uncertainty, and can emulate massive climate model outputs (55,000+ variables) in minutes on a standard laptop using only a handful of training samples.

What Problem Does This Paper Address?

Modeling high-dimensional joint distributions, such as global weather patterns or satellite data, is notoriously difficult. These "spatial fields" are often non-Gaussian (they have "heavy tails" or asymmetries) and nonstationary (their behavior changes depending on where you are on the globe).

Researchers usually face a "pick your poison" scenario:

  1. Classical Spatial Statistics: Methods like Gaussian Processes are mathematically elegant but usually assume the data is Gaussian, which is often false for variables like precipitation.
  2. Deep Learning (GANs/VAEs): These can capture complex patterns but require massive amounts of training data and are "black boxes" that struggle to quantify how uncertain their predictions are.

The authors bridge this gap, targeting the "low-data, high-dimension" regime—for example, when you only have 50–100 expensive climate model runs but need to simulate thousands of new, realistic scenarios.

Methods

The core idea is the Triangular Transport Map. Think of this as a nonlinear version of a Cholesky decomposition. It transforms a complex vector $y$ into a vector of independent standard Gaussian variables $z$.

Key innovations include:

  • Bayesian Nonparametric Priors: Instead of using a fixed formula for the map, the authors model the map's components using Gaussian Processes. This allows the model to "shrink" toward a simple linear map if the data doesn't show strong nonlinearity, preventing overfitting.
  • Maximin Ordering & Sparsity: To handle thousands of locations, they use a "maximin" ordering (picking points that are as far apart as possible) and only consider a few "nearest neighbors" for each point. This makes the resulting map extremely sparse and computationally efficient.
  • Conditional Near-Gaussianity: The authors provide a theoretical justification (based on the Poincaré inequality) for why complex processes look more Gaussian at finer scales. They bake this "scale decay" into their priors, allowing the model to focus its nonlinear "power" on large-scale features.
  • Dirichlet Process Mixtures (DPM): For cases where the errors themselves are weird (e.g., bimodal), they offer an extension using DPMs to allow for even more flexible marginal distributions.

Datasets

The authors evaluate their method on:

  1. Simulated Scenarios: Four synthetic datasets (LR900, NR900, NI3600, NR900B) designed to test linear vs. nonlinear behavior, irregular spacing, and bimodal errors.
  2. Climate Model Output: Log-transformed precipitation anomalies from the Community Earth System Model (CESM) Large Ensemble Project.
  3. Scale: A global grid of $N = 55,296$ locations.
  4. Training Size: Very small ($n < 100$ replicates).

Key Insights and Ideas

  • Nonlinear PCA: The coefficients produced by the map ($z^*$) can be viewed as "Nonlinear Principal Components." The first few coefficients capture the most important large-scale nonlinear structures of the field.
  • The "Aha" Moment on Scale: The paper argues that while a global field might be highly non-Gaussian, once you condition on a few nearby points, the remaining "fine-scale" variation is often nearly Gaussian. This justifies using simpler models for the "details" and complex models for the "big picture."
  • Closed-Form Inference: Because they use conjugate priors, the posterior map is available in closed form. This means you don't need to run expensive optimizations or wait hours for a neural network to converge.

Key Results

  • Accuracy: In low-data settings ($n=20$ to $100$), the proposed nonlin method consistently outperformed standard Gaussian models and deep learning benchmarks (VAEs/GANs) in terms of Kullback-Leibler (KL) divergence and log-scores.
  • Speed & Scalability: The method processed the global climate dataset ($55k+$ variables) in roughly 6 minutes on a single-core laptop. Competing spatial methods were either too slow to run or significantly less accurate.
  • Emulation: The fitted map acts as a "stochastic emulator." It can generate a new, physically realistic global precipitation field in seconds—a task that would normally require a supercomputer.

Limitations and Future Work

  • Unobserved Locations: The current method is designed to model data on a fixed grid. Predicting values at a completely new, unobserved GPS coordinate requires further extensions (though the authors suggest a path forward using "shrinkage" toward parametric covariances).
  • Spatiotemporal Data: While the paper focuses on spatial fields, the authors note that extending this to time-series data (space-time) is a major avenue for future work.
  • Latent Fields: The authors envision using these learned distributions as "priors" for data assimilation, which could improve weather forecasting by better accounting for non-Gaussian uncertainties.

Summary updated

Abstract

A multivariate distribution can be described by a triangular transport map from the target distribution to a simple reference distribution. We propose Bayesian nonparametric inference on the transport map by modeling its components using Gaussian processes. This enables regularization and uncertainty quantification of the map estimation, while still resulting in a closed-form and invertible posterior map. We then focus on inferring the distribution of a nonstationary spatial field from a small number of replicates. We develop specific transport-map priors that are highly flexible and are motivated by the behavior of a large class of stochastic processes. Our approach is scalable to high-dimensional distributions due to data-dependent sparsity and parallel computations. We also discuss extensions, including Dirichlet process mixtures for flexible marginals. We present numerical results to demonstrate the accuracy, scalability, and usefulness of our methods, including statistical emulation of non-Gaussian climate-model output.
BibTeX
@article{katzfu2023scalable,
  author = {Matthias Katzfuß and Florian Schäfer},
  title = {{Scalable Bayesian transport maps for high-dimensional non-Gaussian spatial fields}},
  journal = {Journal of the American Statistical Association},
  year = {2023},
  doi = {10.1080/01621459.2023.2197158},
}
Figure