Robots and other safety-critical controllers often use a Gaussian process (GP), a probabilistic regression model that returns both a prediction and an uncertainty band, to learn the parts of their dynamics nobody has modeled. The uncertainty band only protects the system if the true function stays inside it. The best existing guarantee for the realistic case, where the model's hyperparameters are estimated from little data, rests on a property that holds only for stationary kernels. A kernel is the similarity function that tells the GP how strongly two inputs are correlated, and a stationary kernel is one where that correlation depends only on the distance between the inputs. Real dynamics are not like this. Friction, contact, and saturation make a function smooth in one region and sharp in another. When the kernel is allowed to vary across space, the existing proof no longer holds, and no one has measured whether the guarantee still holds in practice.
This note proposes a way to recover a certified uncertainty envelope for one tractable and widely used class of such kernels: input-warped kernels, which bend the input space before applying an ordinary stationary kernel. The method splits the hyperparameters into two groups. For the global lengthscale, the old monotonicity argument still works in the warped coordinates. For the warp parameters it does not, and those are handled with an explicit covering argument. The proposal pairs this with a pre-registered empirical audit. The audit records how often the stationary argument actually fails on non-stationary data, and whether the new envelope is tighter than simply shrinking a stationary lengthscale until the bound becomes valid. It is scored on the same safety benchmarks where earlier methods were validated.
Core Claim
Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applications builds a high-probability confidence box over hyperparameters and proves an error bound that holds for every GP in the box. The step that makes this computable is a monotonicity fact. For stationary kernels whose Fourier transform is radially non-increasing (Gaussian, Matérn), the posterior variance only shrinks as the lengthscale grows, so the smallest lengthscale in the box gives the worst case. This is what cut uniform-bound violations on Sarcos robot-arm data from 98% for a vanilla GP to 8%. The paper's own open problems note that the argument collapses once the lengthscale depends on the input.
Our observation is that a large part of practical non-stationarity is stationary in disguise. Write an input-warped kernel as
$$k_{\ell,\psi}(x,x') = k_0\!\left(\frac{\phi_\psi(x)-\phi_\psi(x')}{\ell}\right),$$
where $k_0$ is a Gaussian or Matérn profile, $\phi_\psi$ is a low-dimensional, invertible warp with parameters $\psi$ (for example, a monotone spline per input), and $\ell$ is a global lengthscale. In one dimension a warp with slope $1/\ell(x)$ reproduces a spatially varying lengthscale $\ell(x)$. For fixed $\psi$, the kernel is stationary in warped coordinates, so Capone et al.'s monotonicity in $\ell$ carries over exactly. The difficulty is concentrated entirely in $\psi$.
We bound the worst-case posterior variance over the confidence set $\Theta = [\ell_{\min},\ell_{\max}]\times\Psi$ as
$$\sup_{(\ell,\psi)\in\Theta}\sigma^2_{\ell,\psi}(x)\;\le\;\max_{\psi_j\in\mathcal{N}_\delta}\sigma^2_{\ell_{\min},\psi_j}(x)\;+\;L_\Psi(x)\,\delta .$$
Here $\sigma^2_{\ell,\psi}(x)$ is the posterior variance at test input $x$, $\mathcal{N}_\delta$ is a $\delta$-cover of the warp confidence set $\Psi$, and $L_\Psi(x)$ is a computable Lipschitz constant of the posterior variance in $\psi$, derived from the warp's derivatives and the inverse Gram matrix. This covering-plus-Lipschitz step borrows the discretization strategy that On Uniform Error Bounds for Kernel Regression under Non-Gaussian Noise uses over the input space. We apply it over hyperparameter space instead.
The new claim is therefore specific. For warped non-stationary GPs, a certified envelope needs a cover only of the warp parameters, not of all hyperparameters. The number of GP evaluations grows with the covering number of a low-dimensional $\Psi$, and the resulting bound should be substantially tighter than the valid stationary fallback at the same violation rate.
Research Program
The theory has one hard piece beyond the variance envelope, and we flag it as the main risk. The uniform bound also needs a bound on the norm of the true function in the reproducing-kernel Hilbert space, the function space the kernel defines. This norm changes with the warp. Because the warp is invertible, the norm of $f$ under $k_{\ell,\psi}$ equals the norm of $f\circ\phi_\psi^{-1}$ under the stationary kernel. We will bound it uniformly over $\Psi$ using the smallest and largest warp slopes in the set. If that bound turns out to be vacuous for realistic warps, the certified part of the method is limited to the variance envelope, and we will report that as a negative result.
The build has three parts. The first is a confidence set over $(\ell,\psi)$, constructed as in Capone et al. The second is the certified envelope above, computed with a cover over $\Psi$. The third is the envelope's plug-in to a backstepping tracking controller and to stochastic barrier certificates. The barrier-certificate integration follows Error Bounds for Gaussian Process Regression Under Bounded Support Noise with Applications to Safety Certification, which uses a deep feature extractor in front of a stationary kernel but treats the learned features as fixed. Our warp can be read as a small, explicitly uncertain version of that feature map.
The empirical-validation half is an audit that the stationary literature has not done. For each benchmark we fit warped GPs and record the fraction of test points at which the posterior variance is non-monotone in the global scale once the warp is also uncertain. This directly measures how often Capone et al.'s key lemma fails on real data rather than in principle. We also run a bound-violation study that uses held-out data as ground truth. Both analyses are pre-registered: fixed splits, fixed $\delta$, and fixed confidence level.
Baselines
The naive reference point is a vanilla stationary GP whose hyperparameters are point-estimated by maximum likelihood, with the standard uniform bound. On Sarcos this baseline violated its bound 98% of the time, as reported in Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applications.
The strongest existing method is Capone et al.'s robust stationary bound, which reports an 8% violation rate on Sarcos at $N=850$. It is the strongest competitor because it is the only method in the file that certifies bounds under unknown hyperparameters, and because shrinking the stationary lengthscale can always cover non-stationarity by being conservative everywhere. We will rerun it on identical splits, with the same confidence level and the same hyperparameter-search budget, and first check that we reproduce roughly 8% before comparing. Beating it means matching or improving its violation rate while producing envelopes that are clearly narrower on average. The narrowing should show up especially in the smooth regions of the synthetic non-homogeneous functions, and it should translate into lower tracking error or earlier certification in the control tasks.
For bounded-noise control we compare against Reed et al.'s bounds combined with deep kernel learning. On the inverted pendulum they report certification at 99.9% confidence, against 0% for the earlier Chowdhury and Gopalan bound. We run their pipeline with the same noise bound and data. A fully Bayesian GP with Markov chain Monte Carlo over $(\ell,\psi)$ is included because it is the obvious principled alternative, and it is matched to our method on wall-clock budget.
Two ablations isolate the key ingredient. The first fits the warp by maximum likelihood and applies the Capone box only to $\ell$, which tests whether ignoring warp uncertainty is enough. The second takes the maximum over sampled warps without the Lipschitz correction term, which tests whether the certificate matters empirically.
Why not just…?
Why not just use Capone et al.'s stationary robust bound with a small enough lengthscale? This is the real threat, because a sufficiently short stationary lengthscale may well give valid bounds on non-stationary data. The cost is that the envelope is uniformly wide, sized to the roughest region. The comparison at matched violation rate settles the question. If the warped envelope is not meaningfully narrower on data with known non-homogeneity, such as the $\sin(2x)+\sin(6e^x)$ function and its 2D product from Gauss-Legendre Features for Gaussian Process Regression, then the method is not worth its complexity, and we will say so.
Why not just take the maximum variance over a handful of sampled warps? Empirically this may perform almost as well, and the second ablation measures exactly that. Without the Lipschitz term, however, it certifies nothing between samples. The file's safety papers exist because heuristic uncertainty failed in sparse regions, for example the vanilla GP's 98% violation rate on Sarcos. If the sampled maximum shows the same violation rate at lower cost, we will report the certificate as a guarantee that costs little in practice rather than as a performance gain.
Why not just integrate the hyperparameters out with Markov chain Monte Carlo? Capone et al. found that fully Bayesian GPs still underestimated model error on their tracking benchmark and were too slow for fast control loops. We rerun this comparison with warp parameters included, rather than assuming the finding transfers.
Why not just use deep kernel learning as in Reed et al.? A learned feature map is more expressive, but its uncertainty is not represented. Their bound treats the map as known, and Analysis of structured deep kernel networks analyzes such architectures without benchmark validation. Our warp is deliberately low-dimensional so that its uncertainty can be covered. The trade-off in expressiveness against certifiability is measured directly on the pendulum and the 5D car.
Proposed Experiments
We start with the synthetic 1D and 2D non-homogeneous functions, where ground truth is available at every point. On these we measure violation rate, mean and maximum envelope width, and the audited frequency of non-monotone variance, as the training set grows from 100 to 4,096 points.
We then move to Sarcos at $N=850$ using Capone et al.'s protocol, and to the simulated manipulator tracking task, where we report the maximum tracking error relative to the safety threshold. For bounded noise we use Reed et al.'s inverted pendulum and 5D second-order car, reporting the achieved certification probability.
Compute cost is reported against the size of the warp cover. This makes explicit how quickly the method stops being practical as the warp gains dimensions, since the covering number grows exponentially in the dimension of $\Psi$. Capone et al. already hit the same scaling limit with their grid over hyperparameters.
The decisive check is simple. On Sarcos and the synthetic functions, the method must hold violations at or below the stationary robust bound's level while producing narrower envelopes. If it cannot, the audit alone still settles, with evidence rather than proofs, whether stationarity-based safety guarantees survive non-stationary data.
References
- Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applications
- On Uniform Error Bounds for Kernel Regression under Non-Gaussian Noise
- Error Bounds for Gaussian Process Regression Under Bounded Support Noise with Applications to Safety Certification
- Gauss-Legendre Features for Gaussian Process Regression
- Analysis of structured deep kernel networks