Penalized basis models for very large spatial datasets

Mitchell Krock, William Kleiber, Stephen BeckerarXiv 2019 · DOI · arXiv

Abstract

Many modern spatial models express stochastic variation as a basis expansion with random coefficients. While this framework—encompassing low-rank models, multiresolution representations, and stochastic partial differential equations—addresses dimensionality, it often relies on rigid assumptions regarding coefficient dependence. We propose a flexible graphical model family for these coefficients by parameterizing the precision matrix within a penalized likelihood framework. By encouraging sparsity via an $\ell_1$ penalty and employing a majorization-minimization (MM) approach, we establish a computational connection to the graphical lasso. The resulting nonstationary spatial model is adaptable to very large datasets. When applied to heterogeneous climatological datasets, the model recovers physically sensible structures and outperforms the popular LatticeKrig model in information criteria while remaining competitive in predictive accuracy.

Introduction

In the era of high-resolution satellite imagery and global sensor networks, spatial statistics faces a dual challenge: scalability and nonstationarity. Traditional Gaussian process models, such as Kriging, require $O(n^3)$ computational complexity, rendering them unusable for datasets with millions of locations. Furthermore, these models typically assume stationarity—the assumption that spatial correlation depends solely on distance. In reality, geographic processes are often nonstationary; for instance, temperature correlations across mountainous terrain differ significantly from those across open plains.

To address these issues, researchers often turn to basis expansion models, which represent a spatial field as a weighted sum of simpler functions. However, the challenge then shifts to modeling the dependencies between the expansion coefficients. This paper introduces a method to automatically learn these dependencies from data using penalized likelihood and Gaussian Graphical Models (GGMs).

Methodology

The Basis Expansion Framework

The authors represent a spatial process $Z(s)$ at location $s$ as: $$Z(s) = \sum_{j=1}^{\ell} c_j \phi_j(s) + \epsilon(s)$$ where $\phi_j$ are fixed basis functions, $c_j$ are random coefficients, and $\epsilon(s)$ represents measurement error. The innovation lies in the treatment of the coefficient vector $c$. By assuming $c$ follows a GGM, the spatial structure is governed by the precision matrix $Q$ (the inverse of the covariance matrix).

Penalized Likelihood and Sparsity

To ensure both computational efficiency and model interpretability, the authors encourage sparsity in $Q$. A zero entry in the precision matrix implies conditional independence between two basis coefficients, which effectively defines a "neighborhood" structure in the spatial field. They employ an $\ell_1$ penalty on the off-diagonal elements of $Q$ to induce this sparsity.

Computational Optimization

The resulting optimization problem is non-convex. To solve it, the authors utilize a Majorization-Minimization (MM) approach, specifically a Difference-of-Convex (DC) program. A key theoretical contribution of the paper is the demonstration that each iteration of this DC program is equivalent to solving a Graphical Lasso problem. This allows the model to leverage highly optimized solvers like the QUIC algorithm, making it feasible for "very large" spatial datasets.

Empirical Evaluation

Datasets

The model was evaluated using two significant climatological datasets:

  1. Global Ensemble Forecast System (GEFS): Global surface temperatures comprising 65,160 locations over 1,054 time realizations.
  2. TopoWx (Topography Weather): Minimum temperature residuals over the continental United States (4,577 locations), incorporating elevation as a global basis function to capture terrain-induced variance.

Performance and Results

The penalized basis model was compared against LatticeKrig, a standard tool for large-scale spatial data. While both models showed similar predictive performance in terms of Mean Squared Error (MSE) and Continuous Ranked Probability Score (CRPS), the proposed model demonstrated a superior fit relative to its complexity. In the TopoWx analysis, the penalized model achieved an Akaike Information Criterion (AIC) score of 9,043, compared to 12,615 for LatticeKrig.

Beyond numerical metrics, the model successfully captured complex physical phenomena. It identified the variance discrepancy between land and ocean temperatures and recovered "teleconnections"—long-distance correlations in weather patterns, such as the inverse relationship between atmospheric pressures in disparate geographic regions.

Discussion and Future Directions

By bridging the gap between machine learning (Graphical Lasso) and classical spatial statistics, this framework provides a robust path for modeling nonstationarity at scale. The use of sparse precision matrices transforms dense, intractable operations into efficient sparse matrix calculations.

However, several avenues for future work remain. The choice of basis functions (their scale and placement) still requires manual intervention or cross-validation. Additionally, extending the framework to spatiotemporal data—where the graph structure itself may evolve over time—presents a promising frontier for climatological modeling. Finally, addressing data gaps in global datasets (such as missing ocean observations) will be critical for improving the identifiability of global basis functions.

Summary updated

Abstract

Many modern spatial models express the stochastic variation component as a basis expansion with random coefficients. Low rank models, approximate spectral decompositions, multiresolution representations, stochastic partial differential equations and empirical orthogonal functions all fall within this basic framework. Given a particular basis, stochastic dependence relies on flexible modeling of the coefficients. Under a Gaussianity assumption, we propose a graphical model family for the stochastic coefficients by parameterizing the precision matrix. Sparsity in the precision matrix is encouraged using a penalized likelihood framework. Computations follow from a majorization-minimization approach, a byproduct of which is a connection to the graphical lasso. The result is a flexible nonstationary spatial model that is adaptable to very large datasets. We apply the model to two large and heterogeneous spatial datasets in statistical climatology and recover physically sensible graphical structures. Moreover, the model performs competitively against the popular LatticeKrig model in predictive cross-validation, but substantially improves the Akaike information criterion score.
BibTeX
@misc{krock2019penalized,
  author = {Mitchell Krock and William Kleiber and Stephen Becker},
  title = {{Penalized basis models for very large spatial datasets}},
  year = {2019},
  eprint = {1902.06877},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1902.06877},
}
Figure