When a forecast of air pollution, crime counts, or patient vitals changes, a practitioner wants to know why. Does the change come from a slow trend, from a weather variable, or from two weather variables acting together? Does the effect of a driver drift over the months? They also want error bars on each of those answers. Functional ANOVA, a way of splitting a function into a constant, one-variable effects, two-variable interactions and so on, answers the first questions without ambiguity once each piece is forced to average to zero. Gaussian processes (GPs), Bayesian function estimators that return calibrated uncertainty, can be built so that their posterior comes already split into such zero-mean pieces, each with its own credible band. The obstacle is time. The fast way to run GPs on long series rewrites them as linear state-space models solved by a Kalman filter and smoother. That rewrite needs a Markov prior, one where the next state depends only on the current one. The zero-mean constraint integrates over the whole time window, so it breaks the Markov property, and no existing model offers both properties.
This note proposes a fix. Forcing a GP component to have zero time-average is the same as conditioning the GP on the event that its integral is zero. An integral can be tracked by adding one extra state, the running integral, to the Kalman filter. The constraint then becomes a single noise-free pseudo-observation at the end of the training window. This yields an orthogonal, identifiable decomposition of a multivariate forecast into a time trend, driver main effects, time-varying driver effects, and driver–driver interactions. Every piece gets a posterior band, and training cost stays linear in sequence length.
Core Claim
The research file frames the gap between centered ANOVA kernels and state-space GPs as an incompatibility. The kernel centering of ANOVA kernels and RKHS of zero mean functions for model-based sensitivity analysis adds non-local integral terms. The Kalman machinery of Spatio-Temporal Variational Gaussian Processes (ST-SVGP) needs local transitions. We claim the incompatibility is only apparent. For a temporal base kernel $k$ and a time measure $\mathbb{P}$ on the training window $[0,T]$, the centered kernel is exactly a conditional covariance:
$$k_0(t,t') = k(t,t') - \frac{\int k(t,s)\,d\mathbb{P}(s)\int k(t',s)\,d\mathbb{P}(s)}{\iint k(s,s')\,d\mathbb{P}(s)\,d\mathbb{P}(s')} = \mathrm{Cov}\!\left(f(t), f(t') \,\middle|\, \textstyle\int f\,d\mathbb{P} = 0\right),$$
where $f \sim \mathcal{GP}(0,k)$. The identity holds because the subtracted term is the standard rank-one Gaussian conditioning correction for a linear functional. If $k$ is a Matérn kernel with state-space form, we append the state $z(t) = \int_0^t f(s)\,d\mathbb{P}(s)$, which evolves by $dz = f\,d\mathbb{P}$. This keeps the augmented system linear and Markov at the cost of one extra state dimension. Conditioning on $z(T)=0$ is then a single noise-free observation at the final step, and the Rauch–Tung–Striebel (RTS) smoother propagates it backward. For an empirical measure over the observed time stamps, $z$ becomes a running sum and the same argument holds. Centering along a driver coordinate $x_i$ is static. There we reuse the closed-form centering of Additive Gaussian Processes Revisited (the Orthogonal Additive Kernel, OAK), which uses normalizing flows to make the input measure approximately Gaussian.
The model is a truncated decomposition of the target $y(t)$ over time and exogenous drivers $x_1(t),\dots,x_D(t)$:
$$f(t, x) = f_0 + f_t(t) + \sum_i f_i(x_i) + \sum_{i \in S} f_{t,i}(t, x_i) + \sum_{(i,j) \in P} f_{i,j}(x_i, x_j).$$
Here $f_t$ is the pure temporal trend and seasonality, $f_i$ is the static effect of driver $i$, and $f_{t,i}$ is the part of driver $i$'s effect that drifts over time. The interaction $f_{i,j}$ is a static synergy between two drivers, and $S$ and $P$ are the retained component sets. Every component is centered in every argument, so the pieces are mutually orthogonal and their prior variances are Sobol-style importance weights. Each temporal component is Kalman-smoothed. Each static component uses sparse inducing points, which summarize the data with a small set of virtual points, as in OAK.
Research Program
The first deliverable is the integral-state construction and its correctness proof. We will show that the augmented filter reproduces the dense centered-kernel GP to numerical precision on small problems. We will also extend the construction to products: a space–time component $k_0^t \otimes k_0^{x_i}$ is handled by putting the integral state on the temporal factor and closed-form centering on the driver factor, following ST-SVGP's separable state-space form. Non-Gaussian likelihoods, such as Poisson counts, reuse ST-SVGP's conjugate-computation variational inference, a natural-gradient scheme that keeps every update a Kalman step. The integral-state pseudo-observation is Gaussian with zero noise and needs no special handling.
The second deliverable addresses the state-dimension ceiling. ST-SVGP reports that cost grows as $\mathcal{O}(N_t\, d_{state}^3)$ and becomes intractable beyond $d_{state} \approx 50$. Here $N_t$ is the number of time steps and $d_{state}$ is the size of the hidden state. A joint state over all time-varying components would hit that ceiling quickly. We exploit the decomposition's structure in two ways. Driver–driver interactions are kept static, so they add no temporal state, which removes the quadratic-in-$D$ state growth flagged in Gap 2 of the file. Separately, we fit the temporal components with a variational posterior that factorizes across components and update them by backfitting: each component is Kalman-smoothed against the partial residual of the others. The hypothesis is that orthogonality is exactly what makes this factorization cheap in accuracy. Under the training input distribution, centered components have near-zero cross-covariance, so the posterior correlations that mean-field discards should be small. We will measure the gap against the full joint smoother on problems small enough to run both. The quadratic-time Shapley machinery of Exact Shapley Attributions in Quadratic-time for FANOVA Gaussian Processes then applies directly to the static part and returns per-driver Shapley values with variances. The temporal components report their own bands.
The third deliverable is component selection. A Sobol-style ranking from a static OAK fit chooses the candidate sets $S$ and $P$. During training, the marginal likelihood shrinks the prior variances of unneeded components. This responds to the file's note that centered kernels lack built-in sparsity, and it gives a direct test of Gap 4: do forecasting targets need interactions, or is additivity enough as Interpretable Additive Recurrent Neural Networks For Multivariate Clinical Time Series found on clinical data? We will also examine whether the marginal likelihood of the centered model is harder to optimize, a difficulty the centered-kernel paper reports.
Baselines
The naive reference is a seasonal-persistence forecast plus an unconstrained additive state-space GP: one Matérn temporal component and one component per driver, with no centering. This shows what accuracy and what (non-identifiable) decomposition one gets for free. The strongest GP competitor on the scaling question is ST-SVGP itself, run with the same Matérn order, the same spatial or driver inducing points, the same optimizer and iteration budget, and the same GPU. On NYC-CRIME, Spatio-Temporal Variational Gaussian Processes reports an RMSE of 2.79, against 2.94 for SVGP. Beating it means matching its RMSE and test negative log predictive density (NLPD) within seed noise while also returning an identifiable decomposition. We do not expect to beat it on accuracy, and we will not claim to.
The strongest decomposition competitor is OAK with time included as an input feature, run through standard inducing-point SVGP with the same total inducing budget. It is the closest existing method that yields orthogonal components with uncertainty. Beating it means comparable component recovery at sequence lengths where its inducing points must grow with duration, the regime where ST-SVGP shows SVGP becoming infeasible past about 300 steps.
The strongest deep forecasting competitor is STAM from Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation, which reports an MAE of 29.853 on Beijing PM2.5, against 31.267 for DA-RNN (A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction). We will retrain STAM on our splits with a matched hyperparameter search budget. We count landing within a few percent of its MAE as success, because the proposal's value is calibrated decomposition rather than raw accuracy.
The ablation that isolates the key ingredient is post-hoc projection. We fit the unconstrained additive state-space model, then compute the functional ANOVA of its posterior using the same integral states, which applies the identical linear projection to the posterior. Any gain of the full method over this ablation is attributable to centering the prior rather than centering the answer.
Why not just…?
Why not just fit OAK on lagged and time features as a tabular problem? For short series this is a genuine threat and may match us, since OAK already supplies orthogonal components, Sobol indices, and reported strength such as an AUC of 0.865 against 0.852 for SVGP on SUSY. The weakness is the time axis. Resolving localized temporal structure with inducing points requires a number of points that grows with duration. ST-SVGP documents SVGP becoming infeasible past 300 steps and missing pollution spikes on London air quality, and Natalie Maus et al. describe sparse GPs over-smoothing sharp transients under a fixed budget. The scaling experiment below settles where the crossover lies. If OAK-with-time stays competitive at thousands of steps, our contribution narrows to the theory.
Why not just decompose an unconstrained model after fitting? Every fitted function has a unique functional ANOVA, so post-hoc projection is a real threat to the claim that a centered prior is needed. The projection is linear, so it can even yield component bands. What it cannot do is make prior variances mean component importance during training. Without that, the marginal likelihood cannot prune components, and the faster eigenspectrum decay that OAK reports for orthogonal kernels does not apply. The post-hoc ablation above tests whether these advantages produce measurable differences in recovery, sparsity, and NLPD. If they do not, we will report that centering the posterior is sufficient.
Why not just use an attention forecaster with a post-hoc explainer such as TimeSAE? Attention weights in DA-RNN and STAM say where the network looks. They do not separate main effects from interactions, and they come without uncertainty. TimeSAE gives faithful functional-ANOVA-style concept explanations in 4–5 ms per instance, but it is a post-hoc surrogate trained on large data, with no posterior over components. On planted-ground-truth data we will check whether its interaction attributions recover the true synergy terms as well as our model's components do.
Proposed Experiments
The decisive first check is synthetic. We extend the Sobol g-function used in the centered-kernel paper to a time-indexed version with a planted trend, three driver main effects, one time-varying effect, and one driver–driver interaction. Drivers are correlated so that attention-style attributions oscillate between them. Planted components are known exactly, so we can measure component-wise error, Sobol index error, and the coverage of 90% component bands. We also measure the mean-field-versus-joint posterior gap as a function of how strongly the drivers are correlated. If that gap grows sharply with correlation, the factorization argument fails and we fall back to the joint smoother with fewer time-varying components.
The scaling check reruns ST-SVGP's synthetic sequence-length sweep from 300 to tens of thousands of steps. It compares wall-clock time of our model, ST-SVGP, and OAK-with-time under the same hardware. We expect our cost to track ST-SVGP's within a small constant from the extra integral states.
The real-data checks use NYC-CRIME, which has Poisson counts; fANOVA there is over time and the two location coordinates. They also use London air quality PM10, for which ST-SVGP reports no number on Lacuna, so we will run all methods ourselves. Beijing PM2.5 and SML 2010 cover meteorological and building drivers; no reported result was found on Lacuna for SML 2010 beyond DA-RNN outperforming its baselines. Metrics are RMSE or MAE, NLPD, and interval coverage, plus the stability of Sobol indices across random seeds and across rolling training windows.
A final check on Beijing PM2.5 asks whether the retained interaction set includes a temperature–humidity term and whether pruning it measurably hurts NLPD. This turns the additivity question raised by I-RNN into a testable outcome for forecasting rather than an assumption.
References
- ANOVA kernels and RKHS of zero mean functions for model-based sensitivity analysis
- Spatio-Temporal Variational Gaussian Processes
- Additive Gaussian Processes Revisited
- Exact Shapley Attributions in Quadratic-time for FANOVA Gaussian Processes
- Interpretable Additive Recurrent Neural Networks For Multivariate Clinical Time Series
- Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation
- A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
- Approximation-Aware Bayesian Optimization
- TimeSAE: Causal Sparse Decoding for Faithful Explanations of Black-Box Time Series Models