Post-hoc Popularity Bias Correction in GNN-based Collaborative Filtering

Md Aminul Islam, Elena Zheleva, Ren Wang

Abstract

User historical interaction data is the primary signal for learning user preferences in collaborative filtering (CF). However, the training data often exhibits a long-tailed distribution, where only a few items have the majority of interactions. CF models trained directly on such imbalanced data are prone to learning popularity bias, which reduces personalization and leads to suboptimal recommendation quality. Graph Neural Networks (GNNs), while effective for CF due to their message passing mechanism, can further propagate and amplify popularity bias through their aggregation process. Existing approaches typically address popularity bias by modifying training objectives but fail to directly counteract the bias propagated during GNN's neighborhood aggregation. Applying weights to interactions during aggregation can help alleviate this problem, yet it risks distorting model learning due to unstable node representations in the early stages of training. In this paper, we propose a Post-hoc Popularity Debiasing (PPD) method that corrects for popularity bias in GNN-based CF and operates directly on pre-trained embeddings without requiring retraining. By estimating interaction-level popularity and removing popularity components from node representations via a popularity direction vector, PPD reduces bias while preserving user preferences. Experimental results show that our method outperforms state-of-the-art approaches for popularity bias correction in GNN-based CF.

Introduction

Collaborative filtering (CF) models are designed to learn user preferences from historical data. However, real-world data is usually "long-tailed": a few items (the "head") get most of the clicks, while the vast majority (the "tail") are rarely seen. This imbalance creates a significant challenge for recommendation systems, as models tend to over-recommend popular items regardless of individual user interest.

Distribution of interactions between head and tail items across datasets.

Graph Neural Networks (GNNs), the current state-of-the-art for CF, often exacerbate this problem. Their "message-passing" mechanism causes popular items to have a disproportionate influence on their neighbors. Over several layers of the network, the embeddings of users and items get "pulled" toward these popular hubs. The result is a system that over-recommends trending items, ignores niche interests, and creates a feedback loop that diminishes personalization. Existing solutions usually require modifying the training process or loss functions, which can be unstable and expensive to implement in deployed systems.

Post-hoc Popularity Debiasing (PPD)

The proposed Post-hoc Popularity Debiasing (PPD) method strips popularity bias from GNN recommendation models without requiring any retraining. It operates directly on pre-trained embeddings through a three-step process:

  1. Interaction-Level Popularity Estimation: Instead of treating popularity as a static item label, PPD calculates a "popularity score" for every specific user-item interaction. It compares Global Preference (item similarity to all users) against Personalized Preference (item alignment with a user's specific history).
  2. Popularity Direction Construction: For every node (user or item), the model calculates two centroids in the embedding space: one representing the "popularity-driven" signal and one representing the "true preference" signal. The difference between these two defines a Popularity Direction Vector.
  3. Debiasing via Projection: PPD uses vector projection to identify the component of the node's embedding that lies along the popularity direction and subtracts it. This "filters out" popularity-induced noise while leaving underlying preference signals intact.

Because this happens after training, it can be applied to any GNN-based model (like LightGCN or SGL) without changing the original training code.

Experimental Evaluation

PPD was evaluated on three real-world datasets featuring "unbiased" test sets (random item presentation): KuaiRec (highly skewed short-video logs), Coat (online store ratings), and Yahoo! R3 (music ratings).

Key Results

PPD consistently outperformed state-of-the-art debiasing methods (like MACR, PDA, and DAP) across all metrics, including Recall, NDCG, and Hit Ratio.

Performance comparison (Recall@20) between PPD and LightGCN across different GNN layer depths.
  • Extreme Bias Performance: On the KuaiRec dataset, PPD improved NDCG@20 by over 500% and Recall@20 by 186% compared to the best-performing baseline.
  • Layer Depth Stability: While standard GNNs often see performance drops as they get deeper due to over-smoothing and bias amplification, PPD allowed models to benefit from more layers by cleaning the signal at each step.
  • The "Head-Tail" Win-Win: Contrary to the fear that helping "tail" items hurts "head" item accuracy, PPD shows that removing blind popularity signals improves the model's ability to recommend head items to the right people, boosting accuracy across the board.

Conclusion

PPD offers a stable, post-hoc solution to popularity bias in GNN-based recommendation systems. By identifying and projecting out a popularity direction in the embedding space, it achieves significant gains in ranking quality, especially on highly skewed datasets. Future work may extend this geometric projection approach to other forms of bias, such as exposure or position bias, and explore its application in hybrid recommendation systems.

Summary updated

Abstract

User historical interaction data is the primary signal for learning user preferences in collaborative filtering (CF). However, the training data often exhibits a long-tailed distribution, where only a few items have the majority of interactions. CF models trained directly on such imbalanced data are prone to learning popularity bias, which reduces personalization and leads to suboptimal recommendation quality. Graph Neural Networks (GNNs), while effective for CF due to their message passing mechanism, can further propagate and amplify popularity bias through their aggregation process. Existing approaches typically address popularity bias by modifying training objectives but fail to directly counteract the bias propagated during GNN's neighborhood aggregation. Applying weights to interactions during aggregation can help alleviate this problem, yet it risks distorting model learning due to unstable node representations in the early stages of training. In this paper, we propose a Post-hoc Popularity Debiasing (PPD) method that corrects for popularity bias in GNN-based CF and operates directly on pre-trained embeddings without requiring retraining. By estimating interaction-level popularity and removing popularity components from node representations via a popularity direction vector, PPD reduces bias while preserving user preferences. Experimental results show that our method outperforms state-of-the-art approaches for popularity bias correction in GNN-based CF.
BibTeX
@misc{islam2025post,
  author = {Md Aminul Islam and Elena Zheleva and Ren Wang},
  title = {{Post-hoc Popularity Bias Correction in GNN-based Collaborative Filtering}},
  year = {2025},
  eprint = {2510.12959},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.12959},
}
Figure