People's tastes change, and they rarely change everywhere at once. A user may move from action films to romances while keeping the same standards for acting and pacing. Today's systems for learning from drifting feedback mostly handle this by down-weighting old data in the training loss. Right Now, Wrong Then multiplies each pairwise comparison by $\gamma^{\text{age}}$, and DAO-GP decays past observations the same way. This has two costs. It forgets uniformly, so a stale but still-correct preference is discarded as fast as a stale wrong one. It also shrinks the effective amount of data in every training batch, which makes the gradients noisier just when the model most needs to adapt quickly.
We propose to move forgetting out of the loss and into the prior. We model each user's utility as a Gaussian process over item features, user features and time. A Gaussian process is a Bayesian model that treats the unknown utility as a random function whose values are correlated between similar inputs. We make it cheap with a causal nearest-neighbor approximation: each latent utility value conditions only on its $k$ closest earlier neighbors in item–user–time space. Under this structure an old comparison stops mattering only when newer comparisons about similar items exist to replace it. This is the "screening effect" from spatial statistics, applied to time. Because the training objective then keeps every comparison at full weight, recency can be expressed through which terms we sample in each step, with unbiased importance weights. In that form, recency lowers gradient variance instead of raising it.
Core Claim
Represent drift as correlation that decays over time inside a sparse, causally ordered Gaussian-process prior, rather than as exponential loss weights. We claim this yields three things discounted-loss methods cannot. The first is evidence-dependent, localized forgetting: an old comparison loses influence only where recent nearby data screens it off. The second is calibrated uncertainty about the current utility, usable for choosing informative queries after a shift. The third is an unweighted objective that splits into independent local pieces, so each streaming update can spend its batch on the pieces touched by new data. We test the three claims separately. The central empirical test is whether preferences that did not drift survive a drift in a different item category, a failure mode that global discounting predicts and our model predicts it avoids.
The Model
The latent utility is $f(u, x, t)$ for user features $u$, item features $x$ and time $t$. A comparison "user $u$ preferred $x_i$ over $x_j$ at time $t$" has probit likelihood $\Phi(f(u,x_i,t) - f(u,x_j,t))$, following the population-level preference model of Gaussian Process Preference Elicitation. That paper uses a user-by-item product kernel to borrow strength across people. It fits the model with a dense Laplace approximation that scales cubically, and the authors themselves flag the need for sparse approximations. It also assumes preferences are static. We keep its likelihood and its user-sharing kernel and change both the inference method and the treatment of time.
For the sparse prior we use the Vecchia construction in its variational form, the Variational Nearest Neighbor Gaussian Process (VNNGP) of Wu et al. (2022) and Wu et al. (2023). Inducing values $f_m$ are ordered by time, and each conditions on its neighbor set $S(m)$ drawn only from earlier points:
$$p(f_1,\dots,f_M) \approx \prod_{m=1}^{M} p\big(f_m \mid f_{S(m)}\big), \qquad S(m) \subseteq \{m' : t_{m'} < t_m\},\ |S(m)| \le k.$$
This prior has a sparse precision matrix, as in Zilber and Katzfuss (2021) and Moore and Russell (2015). New inducing points attach to the end of the ordering without touching old factors. This gives a principled answer, in our setting, to the ordering question that VNNGP leaves to random permutation.
Neighbors are found in coordinates rescaled by the learned lengthscales. Periodically we rescale the inputs and recompute the neighbor sets. This addresses the failure of Euclidean neighbor search when time and item features have very different scales, which Wu et al. (2022) and Shi et al. (2025) identify as a limitation.
A pairwise comparison couples two latent values, and the file notes this as an unsolved issue for nearest-neighbor variational methods. We handle it by predicting both values from the union of their neighbor sets, which costs $O((2k)^3)$ per pair. Under the Gaussian variational posterior, the utility difference is a one-dimensional Gaussian, so its expected log-likelihood is computed by Gauss–Hermite quadrature rather than sampling.
Research Program
The first strand is the prior itself. We compare separable kernels (item × user × a Matérn kernel in time) against non-separable ones. We also measure empirically the property we care about: the posterior influence of an old comparison at the current time, estimated by removing it and measuring the change in prediction. We check that this influence falls sharply when fresh comparisons exist nearby and stays high when they do not. A stationary time lengthscale is not uniform discounting, because the screening depends on where the data is. This influence analysis is how we show that.
The second strand is gradient variance in streaming updates. The doubly factorized objective of VNNGP is a sum of local terms $\ell_m$, one per inducing point, each bundling its local KL divergence and its attached comparisons. After a warm start at the previous stage's solution, most terms sit near their optimum and only those near new data have large gradients. We sample a batch $B$ of terms with probabilities $p_m$ and use the unbiased estimator
$$\widehat{\nabla} = \frac{1}{|B|}\sum_{m \in B} \frac{\nabla \ell_m}{p_m},$$
whose variance is minimized when $p_m \propto \|\nabla \ell_m\|$. This is the standard importance-sampling result. We set $p_m$ from a cheap proxy: whether new comparisons landed in term $m$'s neighborhood, plus its last observed gradient norm. We add a floor so that changes propagating along the neighbor graph are not starved.
This scheme only pays off if the converged terms have truly small gradients. Roeder, Wu and Duvenaud (2017) show that ordinary reparameterized estimators carry a score-function term whose variance does not vanish at the optimum. Their "sticking the landing" estimator drops that term. Our quadrature likelihood and analytic Gaussian KL avoid it by construction. We will test directly whether this precondition matters, by comparing plain sampled gradients, sticking-the-landing gradients and quadrature, all under the same sampler. We expect the importance sampler to help much less with plain sampled gradients. This contrasts with discounted-loss training, which the file flags as shrinking effective batch size without analyzing the resulting variance.
The third strand is active querying after drift. Once posterior variance is available, an expected-value-of-information query rule in the style of Bonilla et al. (2010) can target the regions where the posterior has widened because recent data contradicts older data.
Baselines
The naive reference points are two static models on the same features. One is a stationary Bradley–Terry scoring model on all data with no forgetting. The other is the same model retrained on a sliding window, with the window length tuned on held-out early stages. These fix the floor and show how much drift is present at all.
The strongest existing method on the preference-drift question is non-stationary DPO from Right Now, Wrong Then. Under a mid-dataset preference flip on its synthetic benchmarks, it reports reward accuracy "above 60–70%", against "~50–55%" for standard DPO. On UltraFeedback-LM it reports an AlpacaEval 2.0 length-controlled win rate of 8.93 against 7.29 for DPO. Our model scores frozen embeddings rather than fine-tuning a language model, so we run two comparisons. The first is a direct comparison with those published figures on TV-HH and UltraFeedback-RM, using the same change points. The second is a controlled comparison: a discounted Bradley–Terry head of equal parameter count, with $\gamma^{T-t_i-1}$ weighting, trained on the same frozen embeddings. It gets the same optimizer, the same number of gradient steps and a tuning grid over $\gamma$ of the same size as our grid over lengthscales. Beating it means higher post-change reward accuracy at equal compute. Beating it also means no loss on comparisons from unchanged regions. Accuracy on those unchanged regions is the measurement on which the claim rests.
For recommendation streams, the strongest competitor is PESO from Continual Low-Rank Adapters. It keeps one evolving low-rank adapter anchored by a KL penalty and reports average gains of 3.7% to 6.3% on Hit@k and NDCG@k over single-evolving and cumulative adapter baselines. No absolute Hit@k or NDCG values were found on Lacuna for PESO, so we will rerun it under its own 60%-plus-four-stages protocol. It is strongest because its penalty already acts as a data-aware filter that adapts only well-supported directions, which is the closest deterministic analogue of our localized forgetting.
Among Gaussian-process competitors we include two. One is a stochastic variational GP with time as an extra input, run with inducing-point counts matched to our wall-clock budget. The other is DAO-GP on the rating-regression version of each stream. For our main uncertainty benchmark, the Sushi dataset, we found no reported normalized-loss values on Lacuna for GP preference elicitation. We will rerun dense GP preference elicitation on subsamples small enough to fit exactly, which also measures what our approximation costs.
The ablations isolate the key ingredients. The first keeps our model but adds exponential loss weights, which tests whether prior-based forgetting is doing the work. The second uses symmetric, non-causal neighbor sets. The third uses uniform rather than importance-weighted batch sampling. The fourth replaces quadrature with plain sampled gradients, with and without sticking the landing.
Why not just…?
Why not just use exponentially discounted DPO or a Bradley–Terry head with a tuned $\gamma$? It is simple and it works under global flips; the 60–70% reward accuracy reported in Right Now, Wrong Then shows that. This is the real threat. If drift in practice is mostly global, a well-tuned $\gamma$ may match us at lower cost. The localized-drift experiment settles it. Global discounting has to trade post-change accuracy against retention of unchanged preferences, and we will plot that tradeoff across $\gamma$. If some $\gamma$ achieves both, our main claim fails.
Why not just a standard inducing-point GP with time as an input? Wu et al. (2022) report that VNNGP with 32 neighbors outperforms an inducing-point model with 1,024 inducing points on the short-lengthscale UKHousing data. Shi et al. (2025) find that inducing-point models need impractically many pseudo-points as structure grows. Abrupt drift is short-lengthscale structure in time. We expect a low-rank model to blur change points, and we will measure recovery time after each change point to check.
Why not just refit on a sliding window? A window is global forgetting with a hard edge. It discards dormant but valid preferences, the "dormant users" failure noted by PESO, and it has no uncertainty with which to choose queries. It is included as a baseline, so this question is answered directly by the unchanged-region accuracy of the windowed model.
Why not just use uniform batches with Adam? Adam rescales gradients but does not remove the variance. Uniform sampling spends most of the batch on terms that are already converged. If the importance sampler does not reduce measured gradient variance and time-to-recovery at equal step counts, we will drop it and report the model with uniform sampling.
Proposed Experiments
Our main testbed is the Sushi dataset of Bonilla et al. (2010), with its 5,000 users, item attributes and user demographics. Sushi has no timestamps, so we inject controlled drift. We reverse preferences within one item style for a random subset of users partway through the stream and leave other styles unchanged, varying the size and abruptness of the shift. The decisive check is accuracy on unchanged-style comparisons after the shift, together with changed-style accuracy, reported against the full $\gamma$ tradeoff curve. The same data supports the active-query test: we count the queries needed to reach pre-drift accuracy after the shift, comparing our expected-value-of-information rule against random queries and against a discounted model paired with a heuristic query rule.
For real drift we use the Amazon Reviews 2023 categories and Yelp under PESO's chronological protocol, with within-user pairs built from ratings, and report Hit@5, Hit@10, NDCG@5 and NDCG@10. For drift over language-model outputs we use TV-HH and UltraFeedback-RM from Right Now, Wrong Then, with items represented by frozen embeddings, and report reward accuracy before and after each change point. The New Yorker caption rankings (yguooo/newyorker_caption_ranking, from Humor in AI) span 365 contests and serve as an exploratory stream in which humor preferences may shift over time. We do not know in advance that they drift, and we will report a negative result if they show none.
Across all streams we log per-step gradient variance, steps-to-recovery after each change point, and calibration of predicted preference probabilities. The proposal succeeds if three things hold at matched compute. Unchanged-region accuracy after localized drift must exceed every point on the discounted baseline's $\gamma$ curve. Post-change accuracy must be no worse than the discounted Bradley–Terry head. The importance sampler must cut gradient variance when quadrature is used and cut it less with plain sampled gradients. If the first test fails, the prior is not doing what we claim, and the rest of the design does not matter.
References
- Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift
- DAO-GP Drift Aware Online Non-Linear Regression Gaussian-Process
- Gaussian Process Preference Elicitation
- Variational nearest neighbor Gaussian process
- Variational Nearest Neighbor Gaussian Process
- Vecchia-Laplace approximations of generalized Gaussian processes for big non-Gaussian spatial data
- Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling
- Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference
- Continual Low-Rank Adapters for LLM-based Generative Recommender Systems
- Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning