When fitting a Gaussian Process (GP) to spatial data, we often expect that collecting more data will naturally lead to more precise estimates of our model parameters, such as the variance ($\sigma^2$) and the spatial range or lengthscale ($\ell$). However, a fundamental mathematical trap awaits depending on how that data is collected.
If you gather data by expanding your horizon to cover a larger and larger territory, your parameter estimates will behave as expected. But if you gather data by sampling more and more densely within a fixed, bounded region, you will find that it is theoretically impossible to separate the variance from the lengthscale, no matter how many billions of data points you collect.
This phenomenon is governed by the distinction between increasing-domain asymptotics and fixed-domain (or infill) asymptotics. Understanding why individual parameters become unidentifiable in fixed domains—and how specific combinations known as microergodic parameters remain perfectly estimable—is crucial for building interpretable and reliable spatial models.
The Two Ways to Scale Data
To understand why parameter estimation can fail, consider two different ways an observer can collect an infinite sequence of spatial measurements:
Increasing-Domain Asymptotics (Expanding the map)
[ . . ] -> [ . . . . ] -> [ . . . . . . ]
Fixed-Domain Asymptotics (Zooming in)
[ . . ] -> [ .. .. ] -> [Paragraph-dense dots]
- Increasing-Domain Asymptotics: The minimum distance between sampling locations remains bounded away from zero, and the spatial domain expands. As shown by Hossein Keshavarz et al., this regime allows for the consistent estimation of all individual covariance parameters, including variance and range. Because you observe the process across vast distances, you can clearly see how the correlation decays to zero, pinning down the true lengthscale.
- Fixed-Domain Asymptotics: The spatial domain is bounded, and the data density increases by filling in the gaps between existing points. This is the standard reality for many spatial applications, such as analyzing remote sensing data over a specific city or training a GP on a fixed experimental dataset.
Under fixed-domain asymptotics, a fundamental confounding occurs. As you sample more densely, you accumulate infinite information about the local, microscopic behavior of the process, but you learn nothing new about its macroscopic behavior.
Why Individual Parameters Confound
For many popular spatial covariance models, such as the Matérn or exponential kernels, the local behavior of the process's paths is entirely determined by a specific mathematical combination of the variance and the scale parameter.
For instance, in a one-dimensional exponential covariance model $K(x, y) = \sigma^2 \exp(-|x-y|/\ell)$, the local roughness of the sample paths as the distance $|x-y| \to 0$ is dictated by the ratio $\sigma^2 / \ell$. If you double the variance $\sigma^2$ (making the process fluctuate more wildly) but simultaneously double the lengthscale $\ell$ (making the fluctuations more stretched out and gradual), the microscopic behavior of the paths remains identical.
As Daira Velandia et al. demonstrate, under fixed-domain asymptotics, the individual parameters $\sigma^2$ and $\ell$ cannot be consistently estimated. However, their product or ratio forms a microergodic parameter that can be recovered with high precision.
Mathematically, two different parameter settings that yield the same microergodic value produce probability measures that are equivalent (mutually absolutely continuous) on any bounded path. No statistical test can ever distinguish between them based on data from that fixed domain, even with infinite samples. As reviewed by Emilio Porcu et al., this means that any attempt to interpret individual variance or scale parameters from a fixed-domain fit is statistically invalid; only their combined ratio carries physical meaning.
The Microergodic Property
A parameter (or a combination of parameters) is defined as microergodic if it can be estimated consistently from a single realization of the random field within a bounded domain.
François Bachoc et al. explore this boundary when evaluating alternative estimation techniques like composite likelihoods. They show that while full maximum likelihood estimators successfully capture microergodic combinations, misspecified or simplified pairwise likelihoods can easily become inconsistent if they fail to respect these parameter constraints.
The microergodic parameter uniquely characterizes the equivalence class of the underlying Gaussian processes, acting as the sole structural feature that the data can actually "see" when zooming in. As Cheng Li et al. show through Bayesian analysis, the posterior distribution for non-microergodic parameters fails to contract to a point, whereas the posterior for the microergodic parameter contracts efficiently, even in lower dimensions where unidentifiability is most severe.
The Illusion of Interpretability in Complex Kernels
The trap of unidentifiability becomes even more pronounced when machine learning practitioners build complex, additive kernels to capture multi-scale structures—for example, adding a "rough" Matérn kernel to a "smooth" Matérn kernel to model data with both high-frequency noise and low-frequency trends.
Additive Mixture: K = Weight_1 * Matérn(Smooth) + Weight_2 * Matérn(Rough)
|
This component dominates
the local behavior!
Jiawen Chen et al. analyzed these single-output additive mixtures and uncovered a striking result: the least smooth component asymptotically dominates the local behavior of the process. Because fixed-domain data only provides information about this local roughness, the individual mixing weights and variance parameters of the smoother components become completely unidentifiable. Only a single aggregate microergodic parameter can be recovered, rendering any structural interpretation of the individual component weights meaningless.
When are individual parameters identifiable?
Are variance and lengthscale always confounded in a fixed domain? Not necessarily. The confounding happens because kernels like the Matérn are locally isotropic and rough, meaning their global behavior cannot be inferred from local paths.
However, if the kernel is holomorphic (infinitely differentiable and analytic), the story changes. Ameer Qaqish et al. proved that for smooth kernels like the Squared Exponential (RBF) and Periodic kernels, individual variances and lengthscales are identifiable. Because holomorphic functions are entirely determined by their behavior in an arbitrarily small neighborhood, knowing the microscopic path behavior allows the model to uniquely extrapolate the global parameters.
Beyond Bounded Euclidean Spaces
The constraints imposed by microergodic parameters extend far beyond standard flat geography:
- Manifolds: When modeling data on compact Riemannian manifolds (like the surface of the Earth or a complex biological shape), the domain is inherently bounded and cannot expand. Didong Li et al. established that microergodic parameters provide the essential mathematical foundation for conducting reliable statistical inference on manifolds, ensuring that valid conclusions can be drawn despite the geometric constraints that prevent full parameter identification.
- Alternative Asymptotics: If your application allows you to collect data over an expanding area, you can bypass the microergodic limitation entirely. For instance, Reinhard Furrer et al. leverage increasing-domain asymptotics to prove that multivariate tapering methods preserve estimation accuracy. Similarly, Hojun You et al. show that spatial regression models with multiplicative errors can achieve the "spatial oracle property"—converging to the true underlying variables as efficiently as if they were known beforehand—provided the sampling domain expands.
- Other Stochastic Systems: This tension between local observation and global parameter collapse is a general feature of stochastic processes on the edge of stability. For example, Mátyás Barczy et al. demonstrate a parallel issue in multi-dimensional critical affine processes, where estimators for immigration or intercept parameters fail to achieve consistency because the system sits on the boundary between stability and explosion, breaking standard laws of large numbers.
Summary
When designing or interpreting Gaussian Process models, remember that the data collection regime dictates what your model can learn. If you are working within a fixed domain, do not trust individual estimates of variance or range; they are statistical ghosts moving in opposition to one another. Instead, focus your inference and interpretations entirely on the microergodic combinations that uniquely define the process's local reality.
Key Foundational Work
On parameter estimation for critical affine processes (2013) — contributed 5 concept elements to this direction.
On the consistency of inversion-free parameter estimation for Gaussian random fields (2016) — contributed 3 concept elements to this direction.
Asymptotic properties of multivariate tapering for estimation and prediction (2016) — contributed 2 concept elements to this direction.