Fixed-Domain Posterior Contraction Rates for Spatial Gaussian Process Model with Nugget

Cheng Li, Saifei Sun, Yichen ZhuJournal of the American Statistical Association 2023 · DOI

TL;DR

This paper tackles the challenge of accurately estimating parameters in spatial Gaussian process models, particularly when data points are densely sampled within a fixed domain. The authors develop a novel Bayesian framework to establish theoretical guarantees for parameter estimation, showing how quickly the uncertainty about key parameters shrinks as more data becomes available. This work is crucial for reliable spatial modeling in fields like environmental science and remote sensing.

What Problem Does This Paper Address?

Spatial Gaussian process (GP) models are powerful tools for analyzing spatially correlated data, but estimating their parameters, especially the covariance function parameters and the "nugget" (representing measurement error or micro-scale variability), can be tricky. A significant theoretical hurdle arises when dealing with "fixed-domain asymptotics," where the spatial domain is fixed but the number of observations increases, leading to highly dependent data. This dependence makes standard statistical inference challenging. Specifically, for certain common covariance functions (like the Matérn), some parameters become unidentifiable in lower dimensions (d=1, 2, 3) under fixed-domain asymptotics, meaning their true values cannot be precisely estimated even with infinite data. This paper aims to provide theoretical guarantees for Bayesian parameter estimation in such complex scenarios, including the presence of regression terms and the nugget effect.

Methods

The core of the paper lies in developing a novel Bayesian theoretical framework to derive posterior contraction rates. This involves adapting the classic Schwartz's consistency theorem for Bayesian inference. Key methodological contributions include:

  • Novel Evidence Lower Bound: They derive a new polynomial lower bound for the denominator of the posterior density, specifically tailored for the strongly dependent data encountered in fixed-domain asymptotics. This is a crucial step that differs from standard approaches for independent data.
  • Higher-Order Quadratic Variation Estimators: To construct "exponentially consistent tests" (a requirement for the theoretical framework), they propose and generalize higher-order quadratic variation (HOQV) estimators for the microergodic parameter and the nugget. HOQV is a frequentist technique that avoids complex optimization and is applicable to irregularly spaced data.
  • General Theoretical Framework: The developed theory is designed to be general, applying to a broad class of stationary covariance functions beyond just the Matérn.

Datasets

  • Simulated Data: The authors conduct extensive simulations to verify their theoretical findings and assess Bayesian posterior predictive performance. These simulations are performed for domain dimensions d=1 and d=2, with different smoothness parameters (v=1/2 and v=1/4) for the Matérn covariance function.
  • Sea Surface Temperature Data: A real-world dataset of sea surface temperature from the Pacific Ocean is used for a case study. This data is collected from remote sensing satellites on a high-resolution grid.

Key Insights and Ideas

  • Bayesian Guarantees for Challenging Scenarios: The paper provides the first theoretical guarantees for Bayesian parameter estimation in fixed-domain GP models with both regression terms and a nugget effect, addressing a significant gap in the literature.
  • HOQV as a Bridge: Higher-order quadratic variation estimators, a frequentist tool, are ingeniously adapted and utilized to build the necessary components for Bayesian theoretical analysis, bypassing difficult analytical challenges related to the likelihood function.
  • Microergodic Parameter is Key: The work reinforces the importance of the "microergodic parameter" (which uniquely determines the equivalence class of Gaussian processes) for parameter estimation in fixed-domain settings, especially in lower dimensions where individual parameters like the range might be unidentifiable.
  • Nugget's Impact: The presence of the nugget parameter significantly complicates the asymptotic behavior of parameter estimators, leading to slower convergence rates compared to models without a nugget. The paper quantifies this impact.
  • Asymptotic Efficiency of Prediction: The simulations suggest that Bayesian posterior prediction remains asymptotically efficient, closely matching the performance of an "oracle" predictor that uses the true (unknown) parameters.

Key Results

  • Posterior Contraction Rates: The paper establishes general theorems for deriving posterior contraction rates for the microergodic parameter and the nugget parameter under fixed-domain asymptotics.
  • Explicit Rates for Matérn: Explicit posterior contraction rates are derived for the isotropic Matérn covariance function, demonstrating how quickly the posterior uncertainty shrinks with increasing sample size. These rates are shown to be polynomial in the sample size, with logarithmic factors.
  • Simulation Validation: Simulation studies confirm that the posterior distributions contract towards the true parameter values as the sample size increases, supporting the theoretical findings.
  • Real Data Application: The methods are applied to sea surface temperature data, showing the practical utility of the developed framework.

Limitations and Future Work

  • Identifiability of Range and Regression Coefficients: The paper acknowledges that for d=1, 2, 3, the range parameter and regression coefficients are generally inconsistent under fixed-domain asymptotics, and thus their posterior contraction is not the primary focus.
  • Smoothness Parameter Estimation: The current theory assumes the smoothness parameter (v) is fixed and known. Extending the framework to estimate v within the Bayesian setting is identified as an important and challenging future research direction.
  • Anisotropic and Nonstationary Processes: The developed theory is primarily for stationary covariance functions. Extending it to anisotropic or nonstationary spatial processes is a significant area for future work.
  • Optimal Rates: While explicit contraction rates are derived, the paper notes that for the microergodic parameter under stratified sampling, the optimal rate remains an open question.
  • Asymptotic Normality: Establishing the asymptotic normality of the posterior distribution for the parameters is suggested as a future research avenue.

Summary updated

Abstract

Spatial Gaussian process regression models typically contain finite dimensional covariance parameters that need to be estimated from the data. We study the Bayesian estimation of covariance parameters including the nugget parameter in a general class of stationary covariance functions under fixed-domain asymptotics, which is theoretically challenging due to the increasingly strong dependence among spatial observations. We propose a novel adaptation of the Schwartz’s consistency theorem for showing posterior contraction rates of the covariance parameters including the nugget. We derive a new polynomial evidence lower bound, and propose consistent higher-order quadratic variation estimators that satisfy concentration inequalities with exponentially small tails. Our Bayesian fixed-domain asymptotics theory leads to explicit posterior contraction rates for the microergodic and nugget parameters in the isotropic Matérn covariance function under a general stratified sampling design. We verify our theory and the Bayesian predictive performance in simulation studies and an application to sea surface temperature data.
BibTeX
@article{li2023fixed,
  author = {Cheng Li and Saifei Sun and Yichen Zhu},
  title = {{Fixed-Domain Posterior Contraction Rates for Spatial Gaussian Process Model with Nugget}},
  journal = {Journal of the American Statistical Association},
  year = {2023},
  doi = {10.1080/01621459.2023.2191380},
}
Figure