Recommender systems store one learned vector per user and per item in an "embedding table". At production scale these tables hold hundreds of millions of rows and no longer fit in accelerator memory. The standard fix is to make items share parameters, either by hashing several items into one row or by building each item's vector from a few shared pieces. Separately, the popularity-bias literature has found where popularity hides in uncompressed tables. It sits mostly in the vector's length: popular items receive more gradient updates and grow longer vectors. The vector's direction stays a reasonably clean signal of who likes the item, so normalizing lengths at test time removes much of the bias.
This proposal argues that parameter sharing breaks that convenient split. When a rarely seen item shares parameters with a blockbuster, the blockbuster's far larger stream of gradients sets the shared vector's direction, and the rare item becomes a near-copy of its popular neighbor. Normalizing lengths cannot undo this, because the damage is in the direction. We propose to measure this "angular" popularity bias directly and to remove it with a cheap redesign. Each item keeps one private number, a per-item scale that absorbs its popularity. Updates to shared parameters are reweighted so that every item, not every interaction, gets a comparable vote on the shared direction.
Core Claim
Our claim is that under compression, popularity bias moves from vector length into vector direction. A compressed model can match full-table accuracy on logged data while its tail items are directionally indistinguishable from popular items they collide with. Two changes recover most of the lost tail accuracy at a fixed memory budget. The first is to separate magnitude from direction, with magnitude stored privately at a cost of one float per item. The second is to equalize each item's influence on shared directional parameters.
Concretely, an item's embedding becomes $\mathbf{e}_i = s_i\,\mathbf{c}_i/\|\mathbf{c}_i\|$. Here $s_i$ is a private scalar and $\mathbf{c}_i$ is produced by any existing sharing scheme, such as quotient-remainder composition, hashing, or graph-community sharing. During training, the gradient reaching a shared row $r$ is
$$ \mathbf{g}_r=\sum_{i:\,r\in\mathcal{R}(i)} n_i^{-\alpha}\sum_{(u,i)\in\mathcal{D}} \nabla_{\mathbf{c}_r}\ell(u,i), $$
where $\mathcal{R}(i)$ is the set of shared rows item $i$ uses, $n_i$ is its training-interaction count, $\mathcal{D}$ is the training log, $\ell$ is the training loss, and $\alpha\in[0,1]$ moves from today's per-interaction weighting ($\alpha=0$) to equal per-item weighting ($\alpha=1$). At inference the scale $s_i$ is dropped or damped, in the manner of the test-time normalization exponent of Test Time Embedding Normalization for Popularity Bias Mitigation.
What would be new has two parts. The first is the measurement: no compression paper in our file evaluates item-side popularity bias or uses unbiased test sets. The second is that the debiasing acts only on the shared parameters. It does not reweight the whole loss.
Why This Gap Is Real
The motivating evidence comes from both literatures. Test Time Embedding Normalization for Popularity Bias Mitigation shows that item vector length correlates with interaction frequency, with Pearson correlation up to 0.83. It also shows that cosine similarity alone separates liked from unliked items across popularity strata, so dividing by $\|\mathbf{e}_i\|^p$ debiases ranking. Adap-τ : Adaptively Modulating Embedding Magnitude for Recommendation reaches a similar conclusion through normalized training with adaptive temperatures. Both papers assume one private row per item, and that assumption is exactly what compression removes.
On the compression side, the shared-parameter schemes all report success on aggregate metrics computed on logged data. Quotient-remainder composition (Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems) stays within 0.3% to 0.7% of full-table loss at 60x compression, and its authors explicitly list frequency imbalance as unaddressed. Gradient-PCA clustering (Clustered Embedding Learning for Recommender Systems) reports +0.6% AUC at 2,650x reduction. Graph-community sharing (GraphHash: Graph Clustering Enables Parameter Efficiency in Recommender Systems) checks only user-side balance. None of these papers asks whether the tail survives.
The mechanism we hypothesize is the one described for feature hashing in Representational Interference in Feature Hashing: heavy features overwrite light ones that share a bucket. The graph-convolution results in How Graph Convolutions Amplify Popularity Bias for Recommendation? and Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System point the same way. Both find that hub nodes contaminate their neighbors, and community sharing writes that contamination directly into the table.
Research Program
The work has three stages. The first is a diagnostic study. We train full-table matrix factorization and LightGCN models with sampled softmax, then compress them with random hashing, quotient-remainder composition, GraphHash, and mixed-dimension allocation at roughly 4x, 16x, and 64x memory reduction. For every popularity decile we report three things on unbiased test data.
The first is ranking accuracy using cosine scores only. The second is angular fidelity: the rank correlation between a compressed model's cosine scores and the full model's cosine scores over sampled user–item pairs in that decile. The third is a "clone rate": the fraction of tail items whose nearest item in the compressed space is a more popular item that shares a row with it.
The hypothesis predicts a specific pattern. Head-decile fidelity should stay high, tail-decile fidelity should drop sharply as compression grows, and test-time normalization should recover length-induced bias in full tables but not in compressed ones.
The second stage builds the method described above on top of each sharing scheme, using the GraphHash and TTEN codebases. The per-item scale costs $N$ floats against the table's $N\times D$, so the overhead is $1/D$. We charge this overhead to the memory budget. The reweighting needs only per-item counts, which are already available. The exponent $\alpha$ is selected on the ordinary biased validation split, which avoids the dependence on clean uniform data that the file flags for MACR and TTEN. The oracle choice made on uniform data is reported separately.
The third stage is analysis. We trace how $\alpha$ trades head accuracy for tail fidelity, and test whether the improvements persist under popularity shift. For the shift tests we use the setting of Invariant Collaborative Filtering to Popularity Distribution Shift.
Baselines
All methods share the same backbone, loss, embedding dimension, and total parameter count. That count includes private scales, private head rows, or projection matrices, whichever a method uses. Every method also gets the same hyperparameter search budget, the same number of trials over learning rate, regularization, and its own debiasing knobs, all selected on the same validation split.
The naive reference is the compressed model scored by plain inner product, with no debiasing. The full uncompressed table, with and without test-time normalization, serves as the ceiling.
The strongest existing compression competitor is GraphHash, which shares rows among members of Louvain communities in the interaction graph. It is the strongest because it is the only scheme in the file shown to beat hashing by a large margin on exactly our retrieval datasets: an average 101.52% Recall and 88.33% NDCG improvement over hashing at over 75% table reduction (GraphHash). Quotient-remainder composition is the strongest collision-free alternative. We run both with the official GraphHash pipeline on Gowalla, Yelp2018, and AmazonBook.
The strongest debiasing competitor is TTEN applied after compression. It is the natural "separation of concerns" answer, and it is the method whose assumption we are testing. Its paper reports outperforming the best baselines by 4.26% to 31.5% in Recall@20 across Gowalla, Yelp2018, and ML-10M. MACR is included as a causal alternative; it reports average HR@20 gains of 153% over MF and 241% over LightGCN on uniform test sets. We found no reported absolute Recall@20 for any of these methods on Lacuna, and no result for any of them on compressed tables. We therefore reproduce all of them ourselves.
Two ablations isolate our ingredients. The first is the private scale alone with $\alpha=0$. The second is reweighting alone without the private scale. We also include a head-private hybrid, in which the most frequent items covering about 80% of interactions get private rows and everything else is shared. This follows the hot/cold split of Accelerating Recommendation System Training by Leveraging Popular Choices, here used as a representation choice rather than a cache.
Beating the competition means one specific result at equal memory. The method must improve tail-half Recall@20 and NDCG@20 on unbiased test data over GraphHash+TTEN and over the head-private hybrid, with an advantage outside run-to-run variance across five seeds. It must also stay within one point of their head-decile and overall Recall.
Why not just…?
Why not just compress and then apply test-time normalization? This is the main alternative, and the diagnostic study is designed to settle it. TTEN's own argument rests on cosine similarity already carrying clean preference signal, which is established only for private-row tables. If compressed models show high tail-decile angular fidelity, TTEN will be enough, and we will report the gap as negligible. If fidelity collapses in the tail while TTEN still recovers length bias in the full model, then normalization has nothing correct to expose.
Why not just give popular items private rows and share only the tail? This is a genuine threat, because the top 0.7% to 2.4% of entries account for roughly 80% of accesses (Accelerating Recommendation System Training by Leveraging Popular Choices), so the private rows are cheap. Our reason to expect it to fall short is that the tail is itself heavy-tailed: once the head is removed, the torso dominates the deep tail inside every shared row. Reweighting addresses dominance at every scale; a single cutoff does not. The hybrid is run as a full baseline. If it matches our method on the bottom deciles, we will conclude that the problem is purely head-versus-tail and recommend the hybrid.
Why not just reweight the whole loss by inverse popularity? MACR's authors note that inverse propensity weights explode for rare items and destabilize training. Whole-loss reweighting also distorts user vectors and dense layers, which never suffered from collisions. Our reweighting touches only shared item rows. We include standard inverse propensity weighting at the same compression ratios, and compare its seed variance and accuracy directly.
Why not just give tail items fewer dimensions instead of sharing? Mixed-dimension allocation, motivated by the frequency-dimension link in Embedding Compression in Recommender Systems: A Survey, avoids collisions entirely. But it confines rare items to a small subspace, which may limit fine angular separation in a different way. It is included in the diagnostic study. If it preserves tail fidelity at equal memory, it becomes the backbone to which we apply the private scale.
Proposed Experiments
The retrieval experiments use Gowalla, Yelp2018, ML-10M, and Amazon-Book. Each is evaluated twice: on the standard random split, and on uniformly sampled test splits following the MACR and TTEN protocols. Yahoo!R3 and Coat provide genuinely randomly exposed test ratings. KuaiRand provides randomly exposed sequential video data. Douban and Tencent cover temporal and popularity shift as used for InvCF.
The metrics are Recall@20 and NDCG@20, overall and per popularity decile; tail-item coverage and the Gini coefficient of exposure in top-20 lists; and the angular-fidelity and clone-rate diagnostics. Every result is plotted against total memory.
Three outcomes would be decisive. The first is the fidelity curve. Our thesis requires tail-decile cosine rank correlation with the full model to fall with compression under hashing, quotient-remainder, and GraphHash, while head-decile correlation stays flat. Our method must restore tail fidelity, and TTEN must not. The second is the clone test. Under GraphHash, the clone rate of tail items should exceed that of random hashing at equal memory, which would show that community sharing actively merges niche items into hubs. Our reweighting should reduce it. The third is the $\alpha$ sweep. Tail accuracy on unbiased data should rise with $\alpha$ while logged-data accuracy stays roughly flat until $\alpha$ nears 1. A flat tail curve would falsify the gradient-dominance mechanism, and an $\alpha$ chosen on biased validation data should land near the uniform-data oracle.
If the diagnostic finds no tail fidelity loss, the paper becomes a useful negative result: post-hoc debiasing is safe to apply to compressed tables.
References
- Test-Time Embedding Normalization for Popularity Bias Mitigation
- Adap-τ : Adaptively Modulating Embedding Magnitude for Recommendation
- Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
- Clustered Embedding Learning for Recommender Systems
- GraphHash: Graph Clustering Enables Parameter Efficiency in Recommender Systems
- Feature hashing for large scale multitask learning
- How graph convolutions amplify popularity bias for recommendation?
- Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System
- Invariant Collaborative Filtering to Popularity Distribution Shift
- Accelerating Recommendation System Training by Leveraging Popular Choices
- Embedding Compression in Recommender Systems: A Survey
- KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos