Recommender systems learn from logs of what users clicked, bought or played. In those logs a few "head" items collect most of the interactions. Models trained on them then over-recommend those items, which produces more of the same logs. Almost every method for correcting this popularity bias measures an item's popularity as one number, its total interaction count, and penalizes the item in proportion. That single number mixes two different things. An item can be popular because people steadily choose it week after week, or because it went viral: a promotion, a front-page slot or a trending moment pushed it in front of many people at once. The second kind of popularity is mostly a record of exposure. The first is closer to evidence of broad, lasting preference. Penalizing both equally is a plausible reason why debiasing so often trades head accuracy for tail accuracy.
We propose to separate the two components using negative binomial regression, the standard statistical model for counts whose variance exceeds their mean. For each item we fit a hierarchical negative binomial model to its interaction counts per time window, controlling for platform-wide traffic and item age. The model's dispersion parameter measures how bursty the item is, and its posterior gives, for every logged interaction, an estimate of how much a burst inflated exposure at that moment. Existing debiasers usually take a propensity: the estimated probability that an item was exposed to the user, used to reweight training interactions. We replace that with a two-part propensity whose steady and burst components are discounted separately. The bet is that the burst component should be removed almost entirely while the steady component should be removed only lightly, and that this weakens the head–tail trade-off.
Core Claim
The claim has two parts, and the first can be checked before anything is built. First, among items with the same total interaction count, the burstier ones keep less of their popularity when exposure is equalized: under a uniform-exposure test split, and in star ratings where available. If this is false, the idea is dead. Second, a debiaser that discounts the burst component strongly and the steady component weakly beats the same debiaser driven by raw counts. It should win on overall ranking quality under unbiased evaluation and, more specifically, lose less head-item accuracy. The gain should disappear when timestamps are shuffled within each item. That shuffle destroys burstiness but leaves counts unchanged, so it isolates dispersion as the active ingredient.
Where This Sits Relative to Prior Work
Negative binomial modeling has entered recommendation once before. Hierarchical Negative Binomial Factorization for Recommender Systems on Implicit Feedback uses a negative binomial likelihood for user–item play counts. It treats the dispersion-related "failure exposure count" as a latent hierarchical variable and makes training scale linearly in the number of nonzero entries. It is the closest method, but it differs from ours in three ways:
- What is modeled. It models user–item counts as the recommender itself. We model item–time counts as a separate exposure model that feeds a debiaser.
- Data it can use. On the binary data typical of debiasing benchmarks, a user–item count is 0 or 1, so per-pair dispersion carries little information. Item–time counts are overdispersed on any timestamped log.
- Debiasing step. It never removes popularity at ranking time.
The social-media studies motivate the core distinction. Influence of Flavors on the Propagation of E-Cigarette–Related Information: Social Media Study fits a hurdle negative binomial model to share counts. Exploring Mental Health Communications among Instagram Coaches fits a negative binomial regression to likes with follower count as a normalizing term. Punishing evil and promoting good uses the same model for repost cascades. All three show that engagement counts are heavily overdispersed, and that useful conclusions require separating content effects from audience size. They use the model only after the fact, to explain engagement; we turn the same separation into a training signal. The Instagram study's use of follower count as an exposure control is the template for our traffic and item-age controls.
On the debiasing side, three groups of methods use raw popularity as their bias signal, and they supply the plug-in points for our propensity:
- Causal subtraction. Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System (MACR) learns a separate item-popularity branch and subtracts its direct effect at test time.
- Contrastive margins and reweighting. Incorporating Bias-aware Margins into Contrastive Loss for Collaborative Filtering (BC Loss) maps a popularity-derived "bias degree" to an angular margin, through a function its authors call too simple. Popularity-Aware Alignment and Contrast for Mitigating Popularity Bias (PAAC) aligns tail items to head anchors and reweights its contrastive loss by popularity.
- Embedding-length correction. Adap-τ shows that an item's embedding length grows with its interaction frequency, and fixes this by normalizing embeddings with an adaptive temperature.
None of these distinguishes two items with equal counts but different burst structure; that is the single change we introduce.
Two papers constrain the design. MACR reports that plain inverse-propensity weighting is hard to tune and high-variance. BC Loss reports head Recall drops of up to 60% under reweighting. So our propensity must be stable for tail items and must not simply sharpen a global penalty.
Overdispersion also causes trouble of its own. Dispersion estimates for items with one or two events diverge, which forces partial pooling across items.
Research Program
The exposure model. For item $i$ in time window $t$, the interaction count $y_{it}$ follows a negative binomial distribution with mean $m_{it} = \mu_i \exp(s_t + g(a_{it}))$. Here:
- $\mu_i$ is the item's steady base rate.
- $s_t$ is a window effect absorbing platform-wide traffic.
- $g$ is a smooth function of item age $a_{it}$, capturing launch and decay.
The variance is $m_{it} + \alpha_i m_{it}^2$, with item-specific dispersion $\alpha_i$. The log rates and log dispersions get hierarchical normal priors, optionally centered on item-category means. This pooling stabilizes estimates for tail items, which is the dispersion-instability problem noted in the count-modeling literature.
The burst multiplier. Under the gamma–Poisson form of the negative binomial, each window has a latent burst multiplier $\nu_{it}$ with mean 1. Its posterior mean given the observed count is
$$\hat{\nu}_{it} = \frac{1/\alpha_i + y_{it}}{1/\alpha_i + m_{it}},$$
which exceeds 1 exactly when the window was busier than the item's steady expectation. A smaller $\alpha_i$ means a steadier item, and its windows are pulled harder toward 1. Every logged interaction $(u, i, t)$ inherits the $\hat{\nu}_{it}$ of its window.
The two-part propensity. The propensity used by a downstream debiaser is
$$p_{uit} \propto \mu_i^{\gamma_s}\, \hat{\nu}_{it}^{\gamma_b},$$
with separate exponents $\gamma_s$ for steady popularity and $\gamma_b$ for burst-driven exposure. Raw-count inverse-propensity weighting is roughly the special case $\gamma_s = \gamma_b$ with $\hat{\nu}$ averaged out. The hypothesis is that the best setting has $\gamma_b$ near 1 and $\gamma_s$ well below it.
We plug this propensity into three hosts without other changes:
- IPS-weighted LightGCN. The weights become $1/p_{uit}$.
- MACR. Its item-popularity branch is trained to predict $\log p_{uit}$ instead of raw counts.
- BC Loss. Its bias degree is computed from the propensity rather than from interaction counts.
Fitting the exposure model is a one-off cost that is linear in the number of item–window cells. This avoids the cost over all user–item pairs that HNBF had to engineer around.
Data. The standard preprocessed splits of Yelp2018, Gowalla and Amazon-Book drop timestamps. We will rebuild them from the raw public releases with the same filtering and report every baseline rerun on our versions. The percentages reported by cited papers are therefore context, not directly comparable numbers.
The datasets play different roles. MovieLens-10M and the news datasets Globo and Adressa carry timestamps natively. The news data are the strongest test, since news popularity is extremely bursty. Last.fm360K adds true repeat counts, which lets us run HNBF as a reference.
Evaluation. We follow the uniform-exposure test protocol used by MACR, PAAC and GSDA: test interactions are sampled with equal probability per item. We add a forward-in-time split.
Metrics are Recall@20 and NDCG@20 overall and separately for head and tail item groups. We also report the head–tail accuracy gap, which PAAC uses as its headline fairness measure.
Baselines
Naive references.
- Untouched backbone. LightGCN, and plain matrix factorization trained with the pairwise ranking loss BPR, with no debiasing. This shows how much popularity bias is present at all.
- Raw-count inverse-propensity weighting. Weights proportional to $(1/\text{freq}_i)^\gamma$, with $\gamma$ tuned. This is the natural point of comparison, because our method is a strict generalization of it.
- Post-hoc subtraction. Each item's log frequency, scaled by a tuned coefficient, is subtracted from the scores at ranking time.
Strongest existing methods. PAAC is run from its official KDD 2024 repository on a LightGCN backbone. It is the strongest competitor because it targets exactly the head–tail gap we care about. On Yelp2018 it reports a 30.2% NDCG@20 gain and an 8.7% Recall@20 gain over its strongest baseline, and it closes the popular–unpopular accuracy gap by 87.5% on Yelp2018 and 34.18% on Gowalla. No reported absolute NDCG@20 value for PAAC was found on Lacuna, so we will establish it ourselves on our rebuilt splits.
BC Loss is run from its official repository. It reports improving tail performance by 576% while also raising head performance by 40% on Tencent, which makes it the main evidence that the trade-off can already be beaten without our idea.
MACR is included because it is the causal-subtraction counterpart. It reports average HR@20 gains of 153% on MF and 241% on LightGCN across its five datasets. GSDA is included on ML-10M, where it reports a 4.18% NDCG@20 gain over its strongest baseline and a 12.7% NDCG@20 gain over SimGCL in the most unpopular item group.
A reference rather than a debiaser: HNBF runs only on Last.fm360K, where play counts give its negative binomial likelihood something to model. No reported absolute result was found on Lacuna for HNBF either.
Fairness rules. Every method gets the same rebuilt splits and backbone embedding size. Each gets the same tuning budget: 40 trials of random search, with the selection metric computed on a uniform-exposure validation split that is disjoint from test. Results are averaged over five seeds.
Ablations of the key ingredient.
- Timestamp shuffle. Our method with timestamps shuffled within each item, which keeps counts and destroys burst structure.
- Single exponent. Our method with $\gamma_s = \gamma_b$ forced.
- Empirical burstiness. $\hat{\nu}$ replaced by each window's raw count over the item mean, with no regression controls and no pooling.
What counts as winning. Our propensity, inside the same host, must beat that host's raw-count version on overall NDCG@20 under the uniform-exposure split. It must also cut the host's head NDCG@20 loss relative to the untouched backbone by at least a third. The timestamp-shuffle ablation must lose most of the gain. Beating PAAC outright with the best host is the stretch target, not the precondition.
Why not just…?
Why not just tune raw-count inverse-propensity weighting more carefully? The single exponent is the problem, not the tuning. One exponent cannot discount viral exposure heavily without equally discounting items people steadily choose. MACR's reports of tuning sensitivity and BC Loss's reports of collapsing head recall are what that would predict. Our method contains this baseline as a special case, and the single-exponent ablation tests it directly. If the best two-exponent setting turns out to have $\gamma_s \approx \gamma_b$, the decomposition adds nothing and we will say so.
Why not just use the empirical variance-to-mean ratio of each item's counts, without a regression? This is the most serious threat for head items, which have enough windows that a raw ratio is already accurate. The regression adds two things:
- Traffic controls. Without them, a site-wide traffic surge or an item's launch week would be scored as a burst.
- Partial pooling. Without it, tail items, with one or two events, get degenerate dispersion estimates.
We therefore expect the empirical-burstiness ablation to match the full model on head items and fall behind on tail items and on news data with strong daily cycles. If it matches everywhere, the right recommendation is the simpler ratio. We would report that, since the dispersion idea, not the regression machinery, is the contribution.
Why not just use recent or time-windowed popularity as the propensity? Recency records when an item was popular, not whether its popularity came in spikes. A steadily chosen catalog item and a past viral hit can have equal recent counts. A forward-split baseline that uses trailing 30-day popularity in place of $p_{uit}$ will show whether burstiness adds anything beyond recency. On news data the two may be nearly collinear, so the decisive comparison there is the matched-mean analysis described below.
Why not just use HNBF, which already puts negative binomial counts into recommendation? HNBF models how often a user consumes an item, which requires repeat-count data and gives nothing to work with on binary logs. It also keeps popularity inside the learned factors rather than discounting it. We run it on Last.fm360K so readers can see whether a better count likelihood alone reduces popularity bias. Its authors show it fits the popularity distribution more faithfully, which is not the same as debiasing it.
Decisive Checks
The first check needs no recommender. On Yelp2018, Amazon-Book and MovieLens-10M we bin items by total training count and, within each bin, split them into high-dispersion and low-dispersion halves. We then test two predictions:
- In the uniform-exposure test split, low-dispersion items have a higher hit rate per exposure.
- Within the same item, interactions logged during high-$\hat{\nu}$ windows carry lower star ratings than interactions in steady windows.
Star ratings are a preference signal the exposure model never sees. Both effects must hold within count bins, or dispersion is just popularity in disguise.
The second check is the main comparison: our propensity plugged into IPS-LightGCN, MACR and BC Loss, against their raw-count versions and against PAAC, with head and tail reported separately on every dataset. We expect the largest gains on Globo and Adressa, where popularity is most bursty, and the smallest on Gowalla, where check-ins at physical places are habitual. That ordering is itself a prediction.
The third check is the timestamp-shuffle control. If the gain survives the shuffle, the improvement comes from something other than burstiness, and the central claim fails regardless of the headline numbers.
References
- Hierarchical Negative Binomial Factorization for Recommender Systems on Implicit Feedback
- Influence of Flavors on the Propagation of E-Cigarette–Related Information: Social Media Study
- Exploring Mental Health Communications among Instagram Coaches
- Punishing evil and promoting good: the moderating role of subject reputation in the effect of emotion on sharing debunking information
- Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System
- Incorporating Bias-aware Margins into Contrastive Loss for Collaborative Filtering
- Popularity-Aware Alignment and Contrast for Mitigating Popularity Bias
- Adap-τ : Adaptively Modulating Embedding Magnitude for Recommendation