Cross-Day Compatibility of Daily Item Embeddings: An Empirical Study of Alignment, Scale Management, and CTR Adaptation

Yicheng WangarXiv 2026 · arXiv

TL;DR

In industrial recommendation systems, item embeddings are often updated daily, but downstream models like Click-Through Rate (CTR) predictors are trained on multi-day windows. This creates a "version mixing" problem where a single item appears in different coordinate systems and scales across different days, hurting performance. This paper demonstrates that combining upstream stabilization (like warm starts and global rescaling) with downstream task-specific encoders significantly improves cross-day compatibility and CTR prediction accuracy.

What Problem Does This Paper Address?

Modern recommendation engines rely on item embeddings generated from behavioral sequences (e.g., via Word2Vec or graph neural networks). To stay current, these embeddings are typically retrained daily. However, downstream CTR models are often trained on several days of logs to capture enough signal.

This creates a fundamental mismatch: the CTR model sees multiple "versions" of the same item. Because embedding training is often stochastic and coordinate systems are non-identifiable, an item's vector might point in a completely different direction or have a vastly different magnitude from one day to the next, even if the underlying user behavior is stable. The authors investigate how to make these daily "snapshots" compatible so that a shared downstream network can effectively learn from them.

Methods

The authors conducted a massive empirical study with 48 different configurations, comparing two embedding families: Word2Vec (skip-gram negative sampling) and an adaptation of XSimGCL (a graph-based contrastive learning method).

They tested three main categories of interventions:

  1. Upstream Stabilization: Using deterministic initialization (ID-init) and post-hoc Orthogonal Procrustes (OP) alignment to rotate daily versions into a common coordinate system.
  2. Training Continuity: Using Warm Starts, where today's training begins with yesterday's weights.
  3. Scale Management: They introduced global state rescaling (warm_norm), which rescales the entire embedding table to a reference mean norm after each day, preventing the "norm explosion" common in warm-started models.
  4. Downstream Adaptation: Using a shared, trainable encoder (a small MLP) to map embeddings into a lower-dimensional space, testing if the CTR task itself can help align the versions.

Datasets

The study uses proprietary industrial data from LY Corporation’s display-advertising operations.

  • Scale: The catalogs contain at least one million items.
  • Logs: Training and testing used at least one million logged ad impressions and clicks.
  • Temporal Split: A rolling window approach where embeddings trained on data through day $t$ are paired with CTR logs from day $t+1$.

Key Insights and Ideas

  • Retrieval $\neq$ CTR Performance: A surprising finding was that better retrieval metrics (like Recall@100) do not always translate to better CTR prediction. For instance, the graph-based XSimGCL had 76.4% higher recall than Word2Vec but actually performed worse on GAUC (a ranking metric for CTR).
  • The Warm Start Paradox: Warm starts provide excellent directional stability (high cosine similarity between days) but lead to massive scale accumulation. Without intervention, the average vector norm grew by over 100% in just eight days.
  • Global vs. Local Rescaling: Managing scale at the global level (warm_norm) was more effective than forcing every item to a unit vector (warm_unit). Preserving the relative differences in norms between items appears to be a valuable signal for the CTR model.
  • Encoders as Aligners: The downstream encoder does more than just compress dimensions. Trained encoders achieved higher cross-day directional agreement than SVD or random projection controls, suggesting the CTR objective naturally encourages the model to "fix" residual drift in the embeddings.

Key Results

  • Alignment Works: Combining ID-initialization and OP alignment increased adjacent-day cosine similarity by 93.4% and improved GAUC by 2.6% relative to a standard Word2Vec baseline.
  • Scale Management is Crucial: The warm_norm strategy (warm starts + global rescaling) achieved the highest overall GAUC increase of 4.0%. As shown in Figure 1c, it successfully constrained norm growth to just 8.2% compared to the 102.5% seen in unmanaged warm starts.
This figure illustrates how different strategies affect the evolution of embedding geometry. Note the massive norm growth in the 'warm' series (blue) in panel c, which is successfully mitigated by the 'warm_norm' strategy (black).
  • Encoder Gains: In the "train-to-test" scenario, CTR-trained encoders improved directional agreement by up to 10.8 percentage points over untrained controls for aligned Word2Vec models.

Limitations and Future Work

  • Industrial Specificity: The results are based on one specific industrial setting and one temporal window; the authors note that results might vary across different catalogs or timeframes.
  • Offline Only: All results are based on offline metrics (GAUC, LogLoss). While these are standard proxies, the paper does not report online A/B test results or "lift."
  • Causality: While the study shows strong correlations between geometric stability and CTR performance, it doesn't fully isolate the causal mechanisms (e.g., how much of the gain is due to scale vs. direction).
  • Future Directions: The authors suggest that exploring how these stabilization techniques interact with more complex downstream architectures (like Transformers) would be a promising next step.

Summary updated

Abstract

Daily item co-occurrence embeddings are paired with next-day behavior without future-behavior leakage. Multi-day click-through rate (CTR) training therefore mixes versions whose directions and norms may differ for the same item. We compare Word2Vec and an item-graph XSimGCL adaptation across initialization, inheritance, post-processing, training constraints, and downstream augmentation. The inventory contains 48 configurations, including 32 common-protocol CTR evaluations. Observed metrics are reported as relative percentage changes from the corresponding Word2Vec baseline (0%), with raw values and exact dataset sizes withheld. Three findings emerge. First, multiple upstream interventions improve compatibility and CTR. ID-init and orthogonal alignment increase cosine by 51.2% and 93.4%, and GAUC by 2.2% and 2.6%, relative to ordinary Word2Vec. Retrieval and geometry remain insufficient downstream surrogates. Second, warm starts perform strongly but accumulate scale. The day-matched norm increase over Word2Vec reaches 102.5% after eight days; daily global rescaling limits it to 8.2% and attains the highest observed GAUC increase, 4.0%. Third, CTR learning through a dimensionality-reducing encoder further adapts to residual drift. Against dimension-matched random-projection, SVD, and untrained-network controls, trained encoders attain the highest cross-day cosine for four of six upstream methods. The other two rank third and second in train-to-test cosine. These industrial results support complementary roles for upstream stabilization, inherited-scale management, and downstream task learning.
BibTeX
@misc{wang2026cross,
  author = {Yicheng Wang},
  title = {{Cross-Day Compatibility of Daily Item Embeddings: An Empirical Study of Alignment, Scale Management, and CTR Adaptation}},
  year = {2026},
  eprint = {2610.36408},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.36408},
}
Figure