When a recommender or a preference-tuned language model sees a user start behaving differently, current methods mostly forget. Non-stationary preference learning discounts old feedback by its age, so a user who has shifted from "mostly work reading" to "mostly parenting reading" loses the value of years of data. This proposal argues that this forgetting is often wasteful. The user's tastes inside each role may not have changed at all. What changed is how much of the user's attention each role currently gets. If the system models each user as a time-varying mixture over overlapping communities, it can treat the two parts differently. A community is a group of users who share a preference pattern, and a user can belong to several. Each community gets its own non-linear utility function, meaning a model of how much its members like an item, learned from everyone's history and changing slowly. Each user gets a few-dimensional vector of membership weights that is tracked quickly. Under this split, a shift in behavior moves a handful of mixture weights instead of discarding data.
The second ingredient addresses a problem that recency weighting cannot solve: telling real drift apart from a burst of noisy feedback. Real systems collect feedback through several channels, such as clicks, ratings, explicit comparisons, and annotations, and each channel has its own unknown noise level. Real drift should move a user's memberships consistently across channels and often across community peers at the same time. Noise is usually confined to one channel. We propose to estimate per-channel noise jointly with membership drift, adapting the layer-weighting idea from multilayer network statistics, so that the model does not treat a corrupted channel as a change of heart.
Core Claim
Preference drift in multi-user, multi-channel feedback is better modeled as fast movement of each user's overlapping community memberships on top of slowly changing, shared non-linear community utilities. Feedback should be weighted by channel reliability rather than only by age. Compared with age-discounting a per-user model, this should recover faster after a shift, use less data to do it, and react less to channel-specific noise bursts. Where drift is genuinely inside the community utilities, it should do no worse. If an unconstrained time-varying user embedding performs just as well as the overlapping-community structure, the claim narrows to the value of separating timescales and estimating channel noise. One ablation is designed to decide that.
How this differs from the closest work
Three lines of prior work each cover part of this. The first is age-based discounting. Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift replaces the static preference model with exponential discounting $\gamma^{T-t_i-1}$ of each training pair. It shows large gains over stationary training under drift. It treats users in isolation, discounts every sample from time $t$ equally regardless of its source, and its best $\gamma$ depends on a drift budget that is rarely known in advance. This paper motivates the proposal, because it shows drift matters, and supplies the strongest baseline. We differ by never discounting the data used to fit the community utilities. Only the user-state estimate forgets.
The second line is online clustering of users, which handles cluster changes but only as hard switches. Online Clustering of Dueling Bandits learns non-linear utilities from pairwise comparisons and clusters users. Its clusters are disjoint, and its preferences are static, which its authors leave as an open problem. Improved Algorithm on Online Clustering of Bandits and Parallel Online Clustering of Bandits via Hedonic Game add split-and-merge and self-healing coalitions. These show that clusters must be able to reform, but they assume linear rewards and one cluster per user. Our membership vector is a soft, continuous version of "switching clusters." A user can drift from 80/20 to 40/60 between two communities without any discrete event.
The third line supplies the method. For overlapping memberships, Aitchison Embeddings for Learning Compositional Graph Representations represents role proportions on a simplex and uses isometric log-ratio coordinates. These coordinates are an invertible map from proportions to ordinary Euclidean space that respects their relative structure. We reuse this so that drift becomes a random walk in an unconstrained space. Incorporating Implicit Link Preference Into Overlapping Community Detection shows that overlapping memberships can be learned from pairwise preference triples with a ranking loss. It is static, single-layer, and bilinear, and we keep its pairwise likelihood. For channel noise, Recovering shared structure from multiple networks with unknown edge distributions proves that averaging graphs with estimated inverse-noise weights beats plain averaging. Global and individualized community detection in inhomogeneous multilayer networks shows that weak layers can borrow strength from clean ones. Both are static and disjoint. We carry their per-layer reliability idea into a streaming preference model.
Finally, A Shared Low-Rank Adaptation Approach to Personalized RLHF splits a shared projection from per-user adapters, which is structurally close to our shared-functions, per-user-weights design. It is static. It also raises the most dangerous simpler alternative: let the per-user part vary in time and skip the community structure.
Model
Each item or response has features $x$. There are $K$ community utility functions $f_k(x)$. These are small neural heads on a shared trunk, so communities can respond non-linearly and differently to feature combinations. User $i$ at time $t$ has memberships $\pi_i(t)$ on the $K$-simplex, written through log-ratio coordinates $z_i(t)\in\mathbb{R}^{K-1}$ that follow a random walk with step scale $q$. A comparison between items $a$ and $b$, observed on channel $c$, has probability
$$P(a \succ b \mid i,t,c) = \sigma\!\left(\frac{\sum_k \pi_{ik}(t)\,[f_k(x_a)-f_k(x_b)]}{\tau_c}\right),$$
where $\sigma$ is the logistic function and $\tau_c$ is a learned noise temperature for channel $c$. A large $\tau_c$ means channel $c$ carries little information per observation, which plays the role of the inverse-variance weights in the multilayer work. Ratings and clicks enter through matching likelihoods with their own temperatures. The community heads are also allowed a slow drift with a much smaller step scale than $q$. This keeps the model correctly specified when communities themselves change, and the ratio of the two scales is estimated rather than fixed.
Inference alternates between two steps. A filter tracks each $z_i(t)$; we will use a low-dimensional particle filter, which is cheap because $K-1$ is small. Stochastic gradient updates fit the $f_k$ and $\tau_c$ on all of history, weighted by the filtered memberships at the time each observation was made. Separating drift from noise follows from the structure. A surprising observation on one channel can be explained either by moving $z_i$ or by that channel being noisy. Moving $z_i$ must also explain the user's other channels, so persistent inconsistency across channels raises $\tau_c$ instead of moving the user.
Research Program
The work has three parts. The first is a controlled synthetic generator. Users have overlapping memberships, communities have non-linear utilities over item features, and drift can be injected in one of two places: into memberships, or into community utilities. The generator also supports corrupted bursts on single channels. This lets us state in advance where the method should win and where it should only tie. The second part is the model and inference code described above, including the ablations that remove each ingredient. The third is evaluation on real logs with timestamps and multiple feedback types, and on a language-model preference drift benchmark. Analysis will report three things: how many observations after a change point each method needs to return to its pre-change accuracy; how much each method's estimate moves in response to corrupted-channel bursts; and whether the learned memberships match planted or annotated groups.
Baselines
The naive reference point is a stationary model with the same non-linear trunk, a per-user embedding, and no time handling, trained on all history. Every drift method must beat it after change points. It also bounds how much drift actually exists in each dataset.
The strongest drift baseline is NS-DPO-style discounting from Right Now, Wrong Then, applied to the same per-user non-linear preference model. Its $\gamma$ will be tuned on held-out data with the same search budget we give our two step scales. Tuning $\gamma$ on the test drift regime would be an unfair advantage, so we report both oracle-tuned and validation-tuned $\gamma$. On UltraFeedback-LM that paper reports a length-controlled win rate of 8.93 against 7.29 for stationary DPO. After preference flips it reports reward accuracy staying above 60–70%, while DPO fell to about 50–55%. Beating it means higher post-change reward accuracy at equal data and compute, and a lower count of observations needed to recover. On stationary segments we must not be worse, since NS-DPO reports no penalty there.
For clustering baselines we use dynamic disjoint clustering with split-and-merge, as in Improved Algorithm on Online Clustering of Bandits. That paper reports 5.94% to 16.24% lower cumulative regret than CLUB on MovieLens 20M and Yelp. We will give it non-linear utilities in the style of Online Clustering of Dueling Bandits, for which no reported number on these benchmarks was found on Lacuna. This isolates hard versus soft membership.
For sequence models, we use the convolutional sequential recommender ConvRec from Multi-Behavioral Sequential Recommendation on the Amazon datasets. That paper reports gains of up to 4.08% in HR@10 and 7.82% in NDCG@10 over ProxyRCA, and gives no absolute values. We rerun it under our splits with its published configuration and an equal tuning budget.
Three ablations isolate the key ingredients. The first gives every channel one shared temperature, which removes channel reliability. The second fits the $f_k$ with the same age discount as the user state, which removes timescale separation. The third replaces the simplex membership with an unconstrained time-varying user embedding fed to the same trunk, essentially a time-varying version of the per-user adapter in P-ShareLoRA. That paper reports 74.65% accuracy on Reddit TL;DR with Llama-3 8B, and its static version serves as the reference on that dataset.
Why not just…?
Why not just tune the discount factor well? Discounting has one parameter. It must trade off forgetting a user's state against forgetting everything that was learned about item quality, and the drift-budget dependence noted in NS-DPO means no single value serves both. Our synthetic membership-drift regime is built to show this. If validation-tuned discounting matches us there, the timescale-separation claim is false, and we will report it as such. In the utility-drift regime we expect discounting to roughly tie with us. That is the honest boundary of the claim.
Why not just let a sequence model learn drift implicitly? This is a real threat on dense logs. ConvRec-style models read recent context directly and need no explicit state. Their weaknesses should appear in two places. They cannot pool a user's old data with peers' data to re-estimate what the user's newly dominant community likes, which matters for sparse users. They have no channel-reliability mechanism, so a corrupted burst in recent context goes straight into their prediction. We will stratify Amazon results by history length and inject channel corruption. If the sequence model wins even for short histories and under corruption, the structured approach is not justified.
Why not just use a time-varying unconstrained user embedding? This is the closest alternative, and the third ablation tests it directly. An unconstrained embedding is more flexible, but nothing ties its coordinates to stable groups. Peer-level evidence of drift, such as many members of one community shifting together, as in cultural drift, has no shared coordinate to act on. We expect the simplex to help mainly on group-correlated drift and for sparse users. If the ablation ties our full model everywhere, we drop the overlapping-community claim and present the contribution as timescale separation plus channel noise estimation.
Why not just collapse channels with equal weights? Equal-weight averaging of heterogeneous layers is provably suboptimal per Levin et al.. In the multilayer CITE-seq study, inhomogeneous multilayer detection reached 5.89% misclustering, against 8.68% and 10.99% for the single modalities. Whether this carries over to streaming preference channels is exactly what the shared-temperature ablation measures.
Proposed Experiments and Decisive Checks
On the synthetic generator we run four regimes: membership drift, utility drift, mixed drift, and channel corruption, each with users having one to three active communities. The decisive check is a crossover pattern. We should clearly beat tuned discounting on recovery time under membership drift, tie under utility drift, and move far less under single-channel corruption. If that pattern fails, the core claim fails.
On real data we use MovieLens 20M and 25M with their timestamps, framing ratings as within-user comparisons and testing on future periods. We also use the Amazon Beauty, Games, Fashion, and Men sequences, with their interaction types treated as channels, reporting HR@10 and NDCG@10. For preference drift in language models we use the time-varying alignment benchmarks from NS-DPO. The most diagnostic of these is 2C NSGO, whose shift from US-centric to Japan-centric opinions is drift at the level of a group. We expect the largest advantage there and report reward accuracy around change points and length-controlled win rate on UltraFeedback-LM. Where feasible, the New Yorker caption ranking data from Humor in AI offers eight years of weekly crowd ratings. These test slow shifts in shared taste. No reported baseline result was found on Lacuna for drift tracking on that dataset, so we would establish one.
Across all settings we report post-change recovery curves, sensitivity to corruption, and, on synthetic data, membership recovery against the planted mixtures.
References
- Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift
- Online Clustering of Dueling Bandits
- Improved Algorithm on Online Clustering of Bandits
- Parallel Online Clustering of Bandits via Hedonic Game
- Aitchison Embeddings for Learning Compositional Graph Representations
- Incorporating Implicit Link Preference Into Overlapping Community Detection
- Recovering shared structure from multiple networks with unknown edge distributions
- Global and Individualized Community Detection in Inhomogeneous Multilayer Networks
- A Shared Low-Rank Adaptation Approach to Personalized RLHF
- Multi-Behavioral Sequential Recommendation
- Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning