From Pairwise to Ranking: Climbing the Ladder to Ideal Collaborative Filtering with Pseudo-Ranking

Yuhan Zhao, Rui Chen, Li Chen, Shuang Zhang, Qilong Han, Hongtao Song

Introduction

Collaborative Filtering (CF) remains a cornerstone of modern recommender systems. Most current CF models rely on implicit feedback and optimize using pairwise loss functions, such as Bayesian Personalized Ranking (BPR). While effective, pairwise methods only approximate a user's true preference hierarchy by comparing two items at a time. This creates a significant performance gap compared to "ideal" models that could theoretically learn from full rankings of the entire item catalog.

The primary challenges in bridging this gap are twofold: the scarcity of explicit ranking data in real-world scenarios and the lack of efficient loss functions capable of handling complex ranking structures. This paper introduces the Pseudo-Ranking Paradigm (PRP) to address these issues by synthesizing ranking data through a novel noise-injection mechanism.

The Pseudo-Ranking Paradigm (PRP)

PRP aims to simulate the "Ideal CF" scenario where a model learns from a ranked list of items. Since full rankings are unavailable, PRP generates "pseudo-rankings" using a noise-injection strategy.

Conceptual comparison between ideal ranking and pairwise learning, illustrating how PRP selects top-K items for training.

The framework consists of three core elements:

  1. Noise-Injected Pseudo-Ranking: The model takes a positive item (interacted with by the user) and creates synthetic variants by adding varying levels of Gaussian noise to its latent representation. By assuming the original item is superior to the low-noise version, which is in turn superior to the high-noise version, a hierarchy is established.
  2. Ranking Loss Function: A specialized loss function ensures that predicted scores are proportional to these synthetic ranks. This effectively decomposes the ranking task into multiple sub-tasks, ensuring a consistent ordering across the list.
  3. Gradient-based Confidence: To mitigate the impact of potentially inaccurate pseudo-rankings, the model monitors "gradient density." Outlier gradients are down-weighted, enhancing the robustness of the training process.

Experimental Evaluation

The authors evaluated PRP across four benchmark datasets: MovieLens (ML-1M), Yelp, Gowalla, and Foursquare. PRP was tested both as a standalone method and as a loss-function "plug-in" for established architectures like LightGCN and NGCF.

Key Results

  • Performance Gains: PRP consistently outperformed state-of-the-art baselines. On the Yelp dataset, it achieved a 35.13% improvement in Recall@10 and a 38.92% boost in NDCG@10 compared to BPR.
  • Efficiency: Despite processing more complex ranking information, PRP demonstrated high training efficiency. It proved significantly faster than complex negative sampling methods like SimpleX, ranking as the second most efficient method among those tested.

Sensitivity Analysis and Limitations

A critical finding of the study is the sensitivity of the model to the ranking length ($k$) and the noise parameters.

Impact of ranking length (k) and noise level (Beta) on Recall@10 and NDCG@10 for Foursquare and Gowalla datasets.

As shown in the sensitivity analysis, performance typically peaks at a ranking length of $k \approx 4$ or $5$. Beyond this point, performance tends to decline. This suggests that while moving beyond pairs is beneficial, the reliability of synthetic rankings diminishes as the list grows longer. Similarly, the noise level (Beta) must be carefully tuned; the authors found that learning user-dependent noise parameters provides better results than static noise.

Conclusion

The Pseudo-Ranking Paradigm (PRP) successfully bridges the gap between pairwise learning and ideal collaborative filtering. By synthesizing ranking signals and employing a robust ranking loss, PRP achieves substantial performance improvements across diverse datasets while maintaining computational efficiency. Future research will focus on scaling the ranking length $k$ and exploring more sophisticated generative models for pseudo-ranking synthesis.

Summary updated

BibTeX
@inproceedings{zhao2025from,
  author = {Yuhan Zhao and Rui Chen and Li Chen and Shuang Zhang and Qilong Han and Hongtao Song},
  title = {{From Pairwise to Ranking: Climbing the Ladder to Ideal Collaborative Filtering with Pseudo-Ranking}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i12.33462},
}
Figure