Personal Popularity Quantiles Set the Debiasing Strength

Recommender systems trained on click logs tend to push the same few popular items to everyone. One of the cheapest known fixes works at prediction time. When a user's vector is multiplied with an item's vector, the item vector is first divided by its length raised to a power $p$. Popular items have long vectors, so dividing them out shrinks their advantage. The catch is that $p$ is a single number shared by all users and found by grid search. Users differ: some mostly watch blockbusters, others mostly seek obscure items. We propose to set $p$ separately for each user, using only simple descriptive statistics of how popular that user's past items were (their median and spread on a log scale). Concretely, each user's $p_u$ is chosen so that the popularity profile of their recommended list matches the popularity profile of their own history. The match is shrunk toward the population profile when a user has little history.

The appeal is that this needs no retraining, no causal graph, no auxiliary network and no fixed head/tail cutoff. It also addresses a tension the literature leaves open. Test-time normalization treats popularity tolerance as global. Work on personal popularity shows that tolerance is strongly individual, but it handles this with much heavier machinery. In two sentences: popularity debiasing should be strong for niche users and mild for mainstream ones. The user's own median and interquartile range of item popularity are enough to choose how strong, per user, at inference time.

Core Claim

Our claim is that a per-user normalization exponent, chosen by matching quantiles of the user's own popularity history, beats the best globally tuned exponent at an equal tuning budget. The comparison is on both standard and uniform-exposure test sets. We expect the gain to concentrate in users whose history departs from the population. We also claim that the spread of a user's history (interquartile range), not just its center, carries usable signal. A binary head/tail proportion of the kind used by calibration methods carries less.

Background and Positioning

Three papers motivate the approach and also supply its main tool. Test Time Embedding Normalization for Popularity Bias Mitigation (TTEN) shows that item frequency and embedding length are strongly correlated (Pearson up to 0.83). It scores items as $e_u^\top e_i / \|e_i\|^p$ and reports Recall@20 gains of 4.26% to 31.5% over prior debiasing baselines on uniform test sets when the model is trained with sampled softmax. Its $p$ is one dataset-wide value; we reuse its scoring rule and code ([TTEN repo, github.com/ml-postech/TTEN]) and make only $p$ personal. Adap-τ : Adaptively Modulating Embedding Magnitude for Recommendation also proves that embedding length tracks frequency, and it does personalize a scaling parameter. Its personalization is driven by each user's training loss. Graph-convolution recommenders distort that loss through degree-driven gradient dominance, as How Graph Convolutions Amplify Popularity Bias for Recommendation? shows. Our signal is instead a direct summary of what the user consumed. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation supplies our main backbone.

Two papers establish that popularity tolerance is personal, and both pay for that with complexity. Debiasing Recommendation with Personal Popularity (PPAC) separates neighborhood-level from global popularity in a counterfactual model. Its count-only baseline, "MostPPop," often ranks second overall, which is strong evidence that personal descriptive counts are informative. Taming Recommendation Bias with Causal Intervention on Evolving Personal Popularity (CausalEPP) tracks the share of a user's interactions that go to popular items over time, inside a backdoor-adjusted causal model. It reports that LightGCN recommends popular items 75.8% of the time against a 43.3% ground truth, and CausalEPP lowers that to 55.5%. We keep their insight and drop the causal graph. We also replace their binary popular/unpopular share with continuous quantiles.

Three lines of work constrain the design. Long-Tail Session-based Recommendation from Calibration (CSBR) is the closest in spirit. It trains a network so that the head/tail share of a recommendation list matches the share in the user's session. It uses a fixed top-20% cutoff and is degenerate for sessions with no tail items, where the target becomes "recommend zero tail items." Our quantile target on log-degree is continuous, needs no training, and for an all-head user still defines a nonzero spread. Shrinkage toward the population profile prevents collapse. Causal Intervention for Leveraging Popularity Bias in Recommendation (PDA) argues that some popularity reflects quality and should be kept; this is why our target is not "flat popularity." Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System (MACR) and Popularity-Aware Alignment and Contrast for Mitigating Popularity Bias (PAAC) are judged on uniform test sets that reward suppressing popularity. We must therefore report both regimes and not pick the one that flatters us. Finally, the descriptive-statistics literature, for example Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range, supports a methodological choice. On heavy-tailed quantities, medians and interquartile ranges are the stable summaries and means are not. That is why we summarize log-degree by quantiles rather than averages.

Method

Let $d_i$ be the training degree of item $i$, and let $z_i = \log(1+d_i)$. For user $u$, let $Q_u(\alpha)$ be the empirical $\alpha$-quantile of $z_i$ over the user's training items. Let $n_u$ be their number of interactions. Let $Q_{\text{pop}}(\alpha)$ be the same quantile pooled over all training interactions. The target profile is a shrunk quantile function.

$$Q^\ast_u(\alpha) = \frac{n_u\,Q_u(\alpha) + m\,Q_{\text{pop}}(\alpha)}{n_u + m}$$

Here $m$ is a single pseudo-count controlling how much a short history is trusted. Let $\hat Q_u(\alpha; p)$ be the $\alpha$-quantile of $z_i$ over user $u$'s top-$K$ list under TTEN scoring with exponent $p$. We set

$$p_u = \arg\min_{p \in \mathcal{P}} \sum_{\alpha \in \{0.25,\,0.5,\,0.75\}} \big(\hat Q_u(\alpha;p) - Q^\ast_u(\alpha)\big)^2,$$

where $\mathcal{P}$ is a grid of eleven values in $[0,1]$. The method then has exactly one tuned global knob, $m$, the same number of knobs as global TTEN.

For uniform-exposure evaluation we add a variant, with $Q_{\text{pop}}$ replaced by the quantiles of a uniform-exposure profile. Choosing which variant to run is a global decision fixed per evaluation regime and declared in advance. Cost is eleven rescorings per user, which is negligible next to training.

Research Program

The work proceeds in three stages, and the first can kill the project.

The first stage is a headroom study. On a validation split we compute an oracle per-user $p$ chosen on that user's held-out items. We then measure how much NDCG@20 the oracle gains over the best global $p$. We also measure how much of the variance in oracle $p$ across users is explained by the median and interquartile range of the user's history, compared with the binary head share alone. If the oracle gain is small, or if history statistics explain almost none of the oracle variation, the idea is not worth pursuing. We would report that as a negative result about personalization of test-time normalization.

The second stage builds the quantile-matching rule above and evaluates it.

The third stage is diagnostic. It slices results by user mainstreamness, measured as the gap between $Q_u(0.5)$ and $Q_{\text{pop}}(0.5)$, and by item popularity decile. The aim is to show where gains come from and whether the rule hurts mainstream users, the "destructive debiasing" failure that PDA warns about.

Baselines

The naive reference is the backbone with plain inner-product scoring ($p=0$), using LightGCN and matrix factorization trained with both pairwise loss and sampled softmax. This shows how much any normalization helps at all.

The strongest direct competitor is global TTEN. It uses the same trained embeddings, with a single $p$ grid-searched on the same validation split over the same eleven-value grid. Its tuning budget is identical to ours because we tune only $m$ over an eleven-value grid. TTEN reports 4.26% to 31.5% Recall@20 gains over prior baselines on Gowalla, Yelp2018 and ML-10M uniform test sets. No reported absolute Recall@20 value for TTEN was found on Lacuna, so we will reproduce it with the released code and treat our reproduction as the bar. Beating it means a statistically significant gain in NDCG@20 and Recall@20 (paired test across users, five seeds) on at least two of three datasets in both evaluation regimes, with no significant loss on the most mainstream user quintile.

The strongest personalized competitor is PPAC. It reports up to 46.8% Recall and 61.9% NDCG gains over the best prior debiasing methods on Gowalla, and its descriptive baseline MostPPop often ranks second. We run both on our splits with their published search ranges, capped at the same number of validation evaluations we use. Beating PPAC is not required for the core claim. Matching it with no training changes would itself be a meaningful result about cost.

We also include Adap-τ, using its closed-form global temperature plus loss-based personal temperature, as the representative of loss-driven personalization. We include PAAC, using its released code at [github.com/miaomiao-cai2/KDD2024-PAAC], as the strongest training-time method on uniform Yelp2018. PAAC reports NDCG@20 +30.2% and Recall@20 +8.7% there over its strongest baseline.

Four ablations isolate the key ingredient. A median-only target drops the interquartile terms. A binary-share target uses the user's top-20% share, matched by the list's top-20% share, which is the CSBR statistic used at test time. A no-shrinkage variant sets $m=0$. A random-$p_u$ control draws per-user values from the empirical distribution of chosen $p_u$, to rule out gains from merely varying $p$.

Why not just…?

Why not just tune one global $p$, as TTEN does? This is the default, and the headroom study is designed to test it directly. If oracle per-user $p$ gains little over the best global $p$, our proposal fails, and we will say so. The prior evidence for heterogeneity is indirect: PPAC and CausalEPP both find that uniform debiasing misfits users with different conformity. It has not yet been measured on the TTEN exponent.

Why not just bucket users by mainstreamness and grid-search a $p$ for each bucket? This is the most serious threat, because it is simple and plausibly captures most of the benefit. We include it as a baseline with five quintile buckets and a separately tuned $p$ per bucket. That costs five times our tuning budget, so we also report it under a matched budget. If bucketing matches quantile matching, the honest conclusion is that a coarse mainstreamness signal is enough. The per-user matching rule would then be unnecessary, though the finding that personalization of test-time normalization helps would survive. We expect quantile matching to win mainly for short-history users, where shrinkage matters, and in its use of spread, which bucketing on a single score ignores.

Why not just rank by personal neighborhood popularity (MostPPop) or use PPAC? MostPPop's strength in Debiasing Recommendation with Personal Popularity shows personal counts are informative. However, it discards the learned embeddings, and PPAC adds counterfactual branches with sensitive weights and a neighborhood size. Our method keeps the backbone's angular preference signal and uses personal statistics only to set one scalar. Both methods are run head to head, so the experiments settle whether that division of labor loses accuracy.

Why not just subtract $\lambda \log d_i$ from scores, with $\lambda$ personalized the same way? This is a fair variant, and we include it, with $\lambda_u$ chosen by the same quantile-matching objective. PDA reports that heuristic popularity re-ranking underperforms its causal approach, but that comparison used global weights. If the personalized subtraction matches personalized normalization, the contribution lies in the selection rule rather than the scoring form. The paper would then be reframed accordingly.

Proposed Experiments

We use Gowalla, Yelp2018 and ML-10M with the uniform-exposure test splits used by TTEN, MACR and PAAC, alongside their standard random splits. Amazon-book is a fourth dataset for robustness. The backbones are LightGCN and matrix factorization, each trained with pairwise and sampled-softmax losses. Metrics are Recall@20 and NDCG@20 overall, NDCG@20 per item-popularity decile, and NDCG@20 per user-mainstreamness quintile. We also report a calibration error: the 1-Wasserstein distance between the log-degree distributions of the recommended list and the user's held-out items.

The first decisive check is the headroom study. The second is whether quantile matching beats bucket-tuned $p$ at a matched budget for users with fewer than twenty interactions. The third is whether the interquartile-range ablation loses a measurable amount relative to the full target, which tests the claim that spread, not just center, matters. The fourth is whether mainstream users are protected: the top mainstreamness quintile should not fall below the $p=0$ baseline on the standard split. If the first check fails, the project ends with a measured negative result.