SCONE: A Novel Stochastic Sampling to Generate Contrastive Views and Hard Negative Samples for Recommendation

Chaejeong Lee, Jeongwhan Choi, Hyowon Wi, Sung-Bae Cho, Noseong Park

TL;DR

This paper introduces SCONE, a novel framework for recommendation systems that tackles data sparsity and negative sampling challenges. SCONE uses score-based generative models to create personalized "contrastive views" and "hard negative samples," leading to significant performance improvements across various datasets.

What Problem Does This Paper Address?

Recommender systems are crucial for helping users discover relevant items, but they often struggle with data sparsity – the fact that users interact with only a tiny fraction of available items. This makes it hard for traditional collaborative filtering (CF) models to learn accurate user preferences. Furthermore, the way negative samples (items a user didn't interact with) are chosen significantly impacts model performance. Randomly selected negative samples are often too easy for the model to distinguish from positive ones, hindering its ability to learn nuanced preferences. Existing methods for generating "hard" negative samples or augmenting data for contrastive learning often have limitations, such as information distortion or a lack of personalization. This paper aims to unify and improve these techniques by generating personalized, high-quality contrastive views and hard negative samples using a novel stochastic sampling approach.

Methods

SCONE builds upon graph-based collaborative filtering (CF) models, specifically using LightGCN as its graph encoder. The core innovation lies in employing score-based generative models (SGMs) for two key tasks:

  1. Contrastive Views Generation: SCONE uses the forward and reverse processes of an SGM to generate two distinct, yet related, "views" of a user or item embedding. The forward process adds noise, while the reverse process learns to denoise and generate a new embedding. These two views are then used for contrastive learning, helping the model learn robust representations even from sparse data. This process is user-item specific, meaning the generated views are tailored to individual users and items.
  2. Hard Negative Samples Generation: SCONE introduces a "stochastic positive injection" method. During the reverse process of the SGM, it injects information from a positive sample (an item the user interacted with) into the generation of a negative sample. This ensures that the generated hard negative samples are "close" to positive ones in the embedding space, making them more challenging and informative for training. This approach aims to generate diverse hard negative samples that cover the embedding space more effectively than previous methods.

SCONE trains the graph encoder and the SGM end-to-end, optimizing a combined loss function that includes both a standard Bayesian Personalized Ranking (BPR) loss (enhanced by hard negative samples) and a contrastive learning loss.

Datasets

The paper evaluates SCONE on six benchmark datasets:

  • Douban
  • Gowalla
  • Tmall
  • Yelp2018
  • Amazon-CDs
  • ML-1M

These datasets represent a range of sizes and sparsity levels, allowing for a comprehensive evaluation of SCONE's performance and robustness.

Key Insights and Ideas

  • Unifying Contrastive Learning and Negative Sampling: The paper demonstrates that contrastive learning and hard negative sampling, while often treated separately, share a common goal: generating tailored samples to improve model learning. SCONE's success in unifying these within a single framework is a significant contribution.
  • Personalized Augmentation with SGMs: Using SGMs allows for generating dynamic, user-item-specific augmented views and hard negative samples. This is a key differentiator from methods that might perturb the graph structure or add uniform noise, which can lead to information loss or distortion.
  • Effectiveness in Sparse Scenarios: SCONE shows particular strength in handling user sparsity and item popularity issues, outperforming baselines significantly for low-activity users and long-tail items. This suggests its approach is well-suited for real-world recommendation scenarios where data is inherently sparse.
  • Improved Representation Uniformity: The paper shows that SCONE leads to better representation uniformity, which is linked to improved diversity and personalization in recommendations.

Key Results

  • State-of-the-Art Performance: SCONE consistently outperforms existing state-of-the-art models (LightGCN, SGL, NCL, SimGCL, DNS, MixGCF) across all six benchmark datasets in terms of Recall@20 and NDCG@20. For example, on the Douban dataset, SCONE improves Recall@20 by 1.70% and NDCG@20 by 2.79% compared to the best baseline (MixGCF).
  • Robustness to Sparsity: SCONE demonstrates superior performance for extremely sparse user groups and low-popularity items, indicating its effectiveness in addressing cold-start users and long-tail items.
  • Ablation Studies: Both contrastive learning and hard negative sampling components of SCONE contribute significantly to its performance, with their impact varying across datasets.

Limitations and Future Work

  • Focus on LightGCN: The current work focuses on LightGCN as the graph encoder. Future work could explore integrating SCONE with more advanced graph-based CF models.
  • Distinction from Sequential Recommendation: The paper notes that SCONE's task differs from sequential recommendation models that also use generative models. Future work could explore adapting SCONE's strengths to sequential recommendation tasks.
  • Advanced Sampling Techniques: The authors suggest exploring more sophisticated techniques, such as positive injection methods from other works (e.g., DINS), and extending SCONE to various recommendation tasks.
  • Hyperparameter Sensitivity: While SCONE performs well, its performance can be sensitive to hyperparameters like the regularization weight ($\lambda_1$) and the weight of stochastic positive injection ($w$). The number of sampling steps ($N$) also needs careful tuning to avoid excessive perturbation.

Summary updated

BibTeX
@inproceedings{lee2025scone,
  author = {Chaejeong Lee and Jeongwhan Choi and Hyowon Wi and Sung-Bae Cho and Noseong Park},
  title = {{SCONE: A Novel Stochastic Sampling to Generate Contrastive Views and Hard Negative Samples for Recommendation}},
  booktitle = {WSDM},
  year = {2025},
  doi = {10.1145/3701551.3703522},
}
Figure