Correlation Horizons That Breathe: Gibbs Kernels with Time-Cycling Lengthscales

When we interpolate a physical field such as air pollution or rainfall from scattered sensors, the key modeling decision is how far apart two readings can be and still inform each other. Gaussian process regression makes this decision explicit through a lengthscale, the distance over which correlation decays. Non-stationary versions let the lengthscale change from place to place, so that a flat plain can be smooth while a mountain ridge stays sharp. In current models, however, this map of lengthscales is frozen in time. Physics says it should not be. At night, a shallow inversion layer traps pollution close to its sources, so correlation between stations should shrink. By afternoon, mixing spreads it across the city, so correlation should grow. Monsoon rain and winter westerly storms over the Himalaya likewise have very different spatial footprints.

We propose a Gaussian process whose lengthscale field changes over time as well as space, written $\ell(x,t)$. The field is expanded in a small fixed basis: smooth spatial functions multiplied by low-order daily and annual sinusoids. Because the temporal part is periodic by construction, the learned "breathing" of correlation carries forward into future days and seasons instead of being fit only to the training window. The model therefore keeps what Gaussian processes do well, namely calibrated uncertainty and a valid covariance, and adds the structured time encoding that recent coordinate networks showed to be essential.

Core Claim

A space-time Gibbs kernel, with lengthscales that are low-order periodic functions of time and smooth functions of space, should beat both of its natural competitors when forecasting forward in time on data with real daily or seasonal cycles. The first competitor keeps lengthscales fixed in time. The second learns lengthscales with an unconstrained neural network. The advantage should show up mainly in predictive calibration (log density and interval coverage) rather than only in point error. The periodic restriction is the key ingredient. Letting lengthscales depend on time is easy. Letting them depend on time in a way that extrapolates correctly is what is missing.

Relation to Prior Work

The mechanism comes from Nonstationary Covariance Functions for Gaussian Process Regression. That paper proved that a kernel built from locally varying lengthscale matrices stays positive semi-definite, meaning it remains a valid covariance, for any positive lengthscale function of the input. Treating time as one coordinate of the input therefore makes $\ell(x,t)$ valid at no extra cost.

Two papers supply the motivation, because they stop exactly short of this step. Learning Non-Stationary Space-Time Models for Environmental Monitoring (NOSTILL-GP) places latent Gaussian processes on spatial and temporal lengthscales, but conditions all three only on location. Its temporal scale therefore varies across space and never across time. Accurate and Scalable Gaussian Processes for Fine-Grained Air Quality Inference combines a spatial Gibbs kernel with a separate periodic kernel for daily cycles. This lets the mean correlation pattern repeat daily, but it does not let the spatial correlation distance itself shrink at night. Kernel Learning for Explainable Climate Science shows that a spatial Gibbs kernel beats both stationary kernels and a two-layer deep Gaussian process on Himalayan precipitation. Its temporal part is still a stationary periodic kernel.

Two neural approaches do allow time-varying kernel parameters, and they are our closest competitors. Neural Non-Stationary Spectral Kernel (Neural-GSM) maps inputs through a multilayer network to mixture weights, lengthscales and frequencies. It was tested on one-dimensional series and tabular data, and the authors report sensitivity to initialization and overfitting. Early Detection of COVID-19 Hotspots Using Spatio-Temporal Data uses a deep network over location and mobility features to set local covariances. Neither constrains how its parameters behave outside the training time window. A network fed raw $t$ has no reason to repeat a daily pattern it saw last month.

The temporal parameterization is taken from A Joint Space-Time Encoder for Geographic Time-Series Data. That work showed Fourier time bases with orthonormality beat direct time input (RMSE 2.704 vs 3.144 on its climate-emulator subset), but its model is deterministic and defines no kernel. We use its lesson inside the covariance rather than in a regression head.

Research Program

Let $z=(x,t)$ be a space-time input with spatial coordinates $x$ and time $t$. The kernel is the multi-dimensional Gibbs form

$$k(z,z') = \sigma(z)\sigma(z')\prod_{d}\sqrt{\frac{2\ell_d(z)\ell_d(z')}{\ell_d(z)^2+\ell_d(z')^2}}\;\exp\Big(-\sum_d \frac{(z_d-z'_d)^2}{\ell_d(z)^2+\ell_d(z')^2}\Big).$$

Here $d$ ranges over the spatial and temporal axes, $\ell_d(z)$ is the lengthscale along axis $d$ at point $z$, and $\sigma(z)$ is a local signal amplitude.

Each log-lengthscale is a truncated expansion

$$\log \ell_d(x,t) = b_d + \sum_{j\le J}\sum_{k\le K} a_{djk}\,\psi_j(x)\,\phi_k(t).$$

In this expansion, $\psi_j$ are orthonormal spatial basis functions: two-dimensional Fourier modes for regional domains, or spherical harmonics for global ones. The $\phi_k$ are a constant plus sines and cosines at the daily and annual periods up to order $K$. The $a_{djk}$ are coefficients, and $b_d$ is a baseline.

We put independent Gaussian priors on the coefficients, with variance decaying in spatial degree and temporal order. This makes the expansion a truncated version of the hierarchical "Gaussian process on the lengthscale" prior of Paciorek and NOSTILL-GP, so the model stays Bayesian and its complexity has a single interpretable dial $(J,K)$.

The model is non-separable, because the spatial lengthscale at a location depends on the hour. It is also forecastable, because every temporal term is periodic. The amplitude $\sigma$ gets a static spatial expansion only. This reflects the unidentifiability Paciorek warns about, where a small lengthscale and a large variance can explain the same data. We test the time-varying amplitude variant separately rather than assume it.

Inference uses sparse variational Gaussian processes, which summarize the data through a small set of learned "inducing" points and cost $O(nm^2)$ for $n$ observations and $m$ inducing points. The inducing points are placed in space-time and initialized by greedy variance reduction. This follows NOSTILL-GP's finding that uniform or random placement fails in volatile regions. The coefficients $a$ are fit by maximum a posteriori estimation within the variational objective. On the smallest dataset we compare this against a Laplace approximation, to check that uncertainty about the lengthscale field matters little for predictive calibration.

Baselines

The naive reference is a separable stationary kernel: Matérn over space times a periodic-times-squared-exponential kernel over time, fit with the same sparse inference and inducing budget. On the Himalayan data, Kernel Learning for Explainable Climate Science reports a stationary one-month-ahead RMSE of 1.1086 against 0.9426 for its Gibbs model. This establishes the gap that non-stationarity alone buys.

The strongest existing Gaussian process on the air-quality benchmark is the composite spatial Gibbs plus local periodic kernel of Accurate and Scalable Gaussian Processes for Fine-Grained Air Quality Inference. It reports PM2.5 RMSE 24.28 on Beijing, ahead of Random Forest (26.68) and the ADAIN attention network (29.39), and RMSE 4.65 on London. We reimplement it with its categorical and relevance-weighting components and give it the same meteorological covariates, inducing count and tuning trials as our model. It is the right competitor because it already handles daily cycles, just not cycles in spatial correlation distance. Beating it means lower negative log predictive density and interval coverage closer to nominal on forward-in-time test blocks, with RMSE no worse. Because our splits differ from the paper's, we compare against our rerun, not the quoted number.

The key ablation is the same Gibbs kernel with lengthscales fixed in time, $\ell(x)$ only. This isolates the time-varying ingredient, and it is the NOSTILL-GP and Patel design class.

The neural competitor is our Gibbs kernel with $\log\ell_d$ produced by a Neural-GSM-style network in the spirit of Neural Non-Stationary Spectral Kernel. We run two input variants, raw $(x,t)$ and the same Fourier time features we use. This separates the benefit of periodic structure from the benefit of the fixed low-order restriction.

The deterministic reference is the Space-Time Encoder of A Joint Space-Time Encoder for Geographic Time-Series Data, made probabilistic by a five-member ensemble with a variance head. It reports RMSE 2.704 on its climate-emulator subset.

All models share splits, covariates and a fixed budget of tuning trials. Uncertainty comes from five seeds.

Why not just…?

Why not just keep the lengthscale static and let the periodic kernel handle daily cycles? This is the Patel design and it is the main thing to beat. A periodic factor lets the covariance between two times repeat. It cannot make the covariance between two stations at the same hour depend on that hour. The static-lengthscale ablation tests directly whether that missing freedom matters. If the empirical check below shows no hour-dependence in station correlations, the answer is that it does not, and the proposal fails honestly on that dataset.

Why not just learn $\ell(x,t)$ with a neural network, as Neural-GSM does? Fed Fourier time features, this is a serious threat and could match us inside the training window. Our bet is that a low-order linear expansion with a decaying prior extrapolates and calibrates better on sparse sensor networks of tens of stations, where Neural-GSM's authors already report overfitting. The two-variant neural baseline settles this. If the network with Fourier inputs ties us on forward-block log density, the contribution reduces to "periodic time features in the lengthscale network," and we would report it that way.

Why not just warp the inputs through the Space-Time Encoder and use a stationary kernel on top? Input warping is simpler and can produce time-varying correlation. Paciorek found, however, that warping distorts global geometry and cannot separately control local lengthscale and variance. We include a warped kernel, stationary on Space-Time Encoder features and trained end to end, as an additional baseline under the same budget. A win for warping would redirect the work toward warped kernels.

Why not just add an ensemble to the Space-Time Encoder for uncertainty? Ensembles give spread but no guarantee of calibration away from data, and no valid covariance for kriging or sensor placement. The ensemble baseline measures whether this matters in practice for coverage at held-out stations.

Proposed Experiments

Synthetic recovery. Draw fields from a known kernel whose lengthscale follows a daily cycle, and vary the cycle amplitude down to zero. We measure recovery error of $\ell(x,t)$ and check that the model does not invent cycles when the true amplitude is zero. This also checks for trade-off between lengthscale and variance.

Air quality. On the hourly Beijing and London PM2.5 data, we first run a model-free test. For each hour of day, we plot empirical station-pair correlation against distance and ask whether the distance at which correlation decays varies by hour. We then train on the first nine months and test on the last three. Metrics are RMSE, negative log predictive density, continuous ranked probability score and 95% interval coverage, evaluated both on held-out times and on held-out stations. Spatially and temporally blocked splits follow Consistent Validation for Predictive Methods in Spatial Settings, which shows that random splits overstate performance.

Himalayan precipitation. On the monthly ERA5 Upper Indus Basin data we use an annual temporal basis only and test one-month and one-season-ahead forecasts. The target is the reported Gibbs result: spatial RMSE 0.271 and log predictive density 0.019, and forecast RMSE 0.9426 Kernel Learning for Explainable Climate Science. We ask whether the monsoon versus westerly regimes appear as a seasonal swing in learned lengthscale.

Global scale. On the climate-emulator temperature subset we use spherical harmonics and a daily basis at 6-hour resolution. This probes whether the method scales to about a million points against the deterministic 2.704 RMSE reference.

Decisive checks. The proposal succeeds if two things hold on forward test blocks. First, the time-varying model beats the static-lengthscale ablation in log density and coverage on the air-quality data. Second, the amplitude of its learned daily lengthscale cycle agrees in sign and phase with the hour-binned empirical correlation distances, with shorter correlation at night. It fails if the neural lengthscale network with Fourier inputs matches it, or if the empirical correlations show no hourly structure.