Recommender systems learn from logged feedback, and that log is shaped by what the system chose to show. Hidden factors that drive both what gets shown and how people respond are called unobserved confounders. When they are present, a model trained on the log learns associations that are partly artifacts of past exposure. Prior work shows that the strongest fixes for this need a small set of randomized interactions, where items were shown regardless of the policy. Those fixes have only been tested on a single static snapshot. Deployed systems are refit every week or month as data drifts, but randomized data is expensive and is usually collected once, if at all. The practical question is whether a deconfounding signal collected once can keep helping long after it was gathered.
We propose to represent the confounding bias as a low-dimensional subspace in parameter space, estimated once. Each later time window is fine-tuned from a shared base model, and that fine-tuning produces an update vector. We project the bias subspace out of each update and then merge the updates. The claim we test is that confounding bias rotates slowly in weight space even while user preferences drift. If that holds, one stale randomized anchor can deconfound many future windows, which retraining a debiasing network on that same stale anchor cannot do. If the subspace rotates quickly, the method fails, and we will measure that rotation directly.
Core Claim and Mechanism
Weight-space merging combines models by adding their parameters rather than averaging their predictions. It works when all models are fine-tuned from a common starting point, because the fine-tuning paths then stay in a shared, roughly linear low-loss region. Tangent Model Composition uses this to treat fine-tuned models as additive perturbations of a base model, at the inference cost of a single model. It also warns that the approximation breaks when fine-tuning moves far from the base. That warning is the main constraint on our design.
Let $\theta_0$ be a base model trained on the earliest logged period. At that same period we hold a small randomized anchor set. Fine-tuning $\theta_0$ on the logged data gives $\theta^{obs}$, and fine-tuning it on the anchor gives $\theta^{anc}$. Repeating this over $m$ bootstrap resamples gives difference vectors $d_i = \theta^{obs}_i - \theta^{anc}_i$. We take the top-$k$ singular directions of these $m$ differences, computed per layer, as a bias basis $B$, with projector $P_B$. For each later window $t$, we fine-tune $\theta_0$ on that window's logged data alone to get an update $\Delta_t$, and form the merged model
$$\theta_T = \theta_0 + \sum_{t=1}^{T} \lambda_t \,(I - P_B)\,\Delta_t ,$$
where $\lambda_t$ are merge weights. We consider uniform weights, recency-decayed weights, and the $\sqrt{t}$ scaling from OPCM. They are chosen only on a chronologically later validation slice.
Three properties distinguish this from prior work. The projection removes a confounding direction, not the directions of earlier tasks. The anchor is used once and only as a direction, never as a label target in later windows. Merging lets old windows contribute durable signal without retraining on pooled data.
Relation to Prior Work
Two lines of work motivate the gap.
On the confounding side, Removing Hidden Confounding in Recommendation shows that propensity-based debiasing stays biased even with infinite data when confounders are hidden. Its residual propensity and imputation networks (Res-IPS and Res-DR) are calibrated on as little as 2% randomized data. That work assumes one static observational distribution paired with one static anchor. FCSRec models time-varying confounders such as season and mood inside a normalizing flow, so adapting to new periods means retraining that architecture. The research file notes that most deconfounding work treats the confounding mechanism as stationary.
On the drift side, Guo et al.'s MIMIC-IV benchmark found that invariance methods such as Invariant Risk Minimization did not beat plain training on pooled history. Forcing time-invariance threw away genuinely predictive signal. We take this as a constraint: we do not force every direction to be time-invariant, only a small, explicitly estimated bias subspace. EMDOT and Simonetto et al. show that pooled-history and sliding-window retraining each fail in different regimes, and that evaluations ignoring labeling and deployment delays overstate gains by up to 39.86%.
We reuse methods from continual merging. MagMax picks the largest-magnitude update per parameter. OPCM projects incoming updates orthogonal to earlier ones via a singular value decomposition. Both were evaluated only on clean, discretely staged image tasks with no exposure bias. Ours is the first test of either on chronologically split observational feedback.
Finally, Meng et al. found that rankings of 230 recommender variants under a global temporal split versus leave-one-last splits agree only at Kendall's τ = 0.5284. That result dictates our evaluation protocol: everything is split strictly by time.
Research Program
The work has three parts.
The first is a controlled semi-synthetic testbed built on KuaiRec. Its small matrix is 99.6% dense, with 1,411 users and 3,327 items collected under forced exposure. That density lets us simulate logged data by sampling exposures from a policy that depends on a hidden user-item factor. We then make two things drift independently across simulated windows: true preferences, and the confounder's effect on exposure. With full ground truth, we can measure two quantities directly. One is the principal angle between the bias subspace estimated at window 0 and the true bias subspace at window $t$. The other is the debiasing error as a function of both drift rates. This maps where a one-time anchor stays valid.
The second part is the real-data test. Logged interactions from the KuaiRec big matrix are ordered by interaction timestamp and cut into chronological windows. Before committing to this design, we will verify that the timestamps are fine-grained enough to support several windows. Small-matrix interactions from the earliest period, subsampled to match the 2% anchor budget of Res-DR, form the anchor. Small-matrix interactions from later periods, held out, form the unbiased test set.
The third part is a set of geometric diagnostics. We measure the loss barrier along the line between $\theta_0$ and each $\theta_0 + \Delta_t$, which checks whether the window updates stay in the near-linear regime that merging needs. We also measure how much of each $\Delta_t$'s norm falls inside $B$. The backbones are matrix factorization and a small two-tower model, all started from the same $\theta_0$, so permutation symmetry is not an issue.
Baselines
The naive reference point is the base model $\theta_0$, frozen after the first period. Next to it is plain matrix factorization retrained on all logged history up to each cutoff. In the static setting, Removing Hidden Confounding in Recommendation reports this at AUC 0.820 on KuaiRec. That number comes from their non-chronological protocol, so we rerun it under our split rather than quote it as a target. Pooled history is the baseline that Guo et al. found hardest to beat under temporal shift. The sliding-window variant, retrained only on the most recent window, covers the fast-drift regime highlighted by EMDOT.
The strongest competitor is Res-DR, which reports AUC 0.854 on KuaiRec against 0.827 for ESCM² in the same paper. For a fair comparison, Res-DR gets exactly the same one-time early-period anchor, the same backbone, and the same hyperparameter search budget. It is refit at every window on pooled logged history, using that stale anchor for its consistency loss. We consider our method to beat it only if it shows a statistically significant AUC advantage on the later-period unbiased test set, averaged over windows and seeds, that widens as the gap since anchor collection grows. Matching Res-DR on the first window is expected and is not evidence either way.
The merging baselines are uniform averaging of the window models, MagMax, and OPCM, each run on the same $\Delta_t$ without the bias projection. These isolate whether any gain comes from merging alone.
Three ablations isolate the key ingredient. The first projects out a random subspace of the same rank $k$. The second subtracts the mean bias vector, as in task-vector negation, instead of projecting. The third refreshes the anchor at every window. It is an oracle that needs data a deployed system would not have, and it gives an upper bound.
All methods see identical windows. All evaluations shift the training cutoff by a fixed label delay, following Simonetto et al.. Model selection uses only data that precedes each test window.
Why not just…?
Why not just retrain Res-DR each period with the anchor you already have? This is the real threat, and we test it head-on. Res-DR fits its residual networks to anchor labels. Once preferences drift, a stale anchor pulls the model toward outdated preferences as well as toward unbiased exposure. Our method transfers only a direction, so later windows' own data still determine preference content. If stale-anchor Res-DR matches us at every time gap in both the simulation and on KuaiRec, the weight-space approach adds nothing, and we will report that.
Why not just pool all history, as Guo et al. found works best? Pooling was the winner in Guo et al., but that setting had no exposure confounding. The hidden-confounding analysis shows pooled logged data stays biased however large it grows. More pooling should therefore improve ranking within the biased log without closing the gap on unbiased later data. The pooled baseline measures exactly this.
Why not just average the window checkpoints, or run OPCM, with no confounding step? Plain merging may already cut forgetting relative to sliding windows, as it does in MagMax. Gains of that kind say nothing about bias. Comparing the projected and unprojected versions, plus the random-subspace control, separates what merging contributes from what removing the bias direction contributes.
Why not just recalibrate probabilities over time? The clinical systematic review found that recalibration repairs calibration but not ranking once feature-outcome relationships change. Confounding distorts ranking, so we report AUC and calibration separately and expect recalibration to move only the latter.
Decisive Checks
The idea stands or falls on three outcomes.
First, in the KuaiRec simulation, the angle between the window-0 bias subspace and the true later subspace must stay small at realistic confounder-drift rates. When we push the drift rate high enough that the angle grows, the method's advantage must disappear. This is the stated failure condition, and observing it confirms the mechanism rather than undermining it.
Second, on real chronological KuaiRec windows, the projected merge must beat stale-anchor Res-DR at long anchor-to-test gaps under the same anchor budget. It must also beat the random-subspace ablation, which shows the estimated direction matters and not just any reduction in rank.
Third, the interpolation loss barriers must stay low. If they are high, the window updates have left the near-linear regime that Tangent Model Composition warns about, and any result would be uninterpretable. In that case we would restrict merging to adapter-sized updates.
A positive result on all three would show that a single costly randomized experiment can keep deconfounding a drifting system at the cost of one model update per window.
References
- Tangent Model Composition for Ensembling and Continual Fine-tuning
- Merging on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging
- Removing Hidden Confounding in Recommendation: A Unified Multi-Task Learning Approach
- Flow-based Time-aware Causal Structure Learning for Sequential Recommendation
- Evaluation of domain generalization and adaptation on improving model robustness to temporal dataset shift in clinical medicine
- Evaluating Model Performance in Medical Datasets Over Time
- On the Impact of Industrial Delays when Mitigating Distribution Drifts: An Empirical Study on Real-World Financial Systems
- MAGMAX: Leveraging Model Merging for Seamless Continual Learning
- Exploring Data Splitting Strategies for the Evaluation of Recommendation Models
- KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems
- Systematic Review of Approaches to Preserve Machine Learning Performance in the Presence of Temporal Dataset Shift in Clinical Medicine