Efficient Heterogeneous Collaborative Filtering without Negative Sampling for Recommendation

Chong Chen, Min Zhang, Yongfeng Zhang, Weizhi Ma, Yiqun Liu, Shaoping Ma

Abstract

Traditional recommender systems rely on negative sampling to handle implicit feedback, a process that is often biased and computationally sensitive. Furthermore, many models fail to exploit the rich, heterogeneous nature of user behaviors, such as viewing or adding items to a cart before a purchase. This paper introduces Efficient Heterogeneous Collaborative Filtering (EHCF), a framework that eliminates negative sampling by optimizing over the entire dataset with linear complexity. By incorporating a transfer-based prediction layer, EHCF models the sequential dependencies between different user behaviors as translations in a vector space. Experimental results on large-scale datasets like Taobao and Beibei demonstrate that EHCF significantly outperforms state-of-the-art methods in both accuracy and training efficiency, achieving up to a 23x speedup.

Introduction

Most modern recommendation algorithms utilize implicit feedback, where only positive interactions are observed. To train these models, researchers typically employ Negative Sampling (NS), which treats a subset of unobserved items as negative signals. However, NS is inherently biased and its performance is highly sensitive to the sampling ratio.

Beyond the challenges of sampling, users leave a trail of diverse interactions—views, clicks, and cart additions—before making a purchase. These heterogeneous behaviors are often ignored or treated as independent tasks. EHCF addresses these gaps by providing an efficient way to learn from the entire dataset while capturing the natural funnel of user intent.

Methodology

Fast Whole-Data Optimization

EHCF's primary innovation is a mathematical rearrangement of the loss function that allows the model to consider every unobserved item as a negative signal without the $O(Users \times Items)$ computational burden. By leveraging the sparsity of positive interactions and rearranging nested sums, the authors reduce the complexity to a linear scale relative to the number of users and items. This ensures stable convergence and removes the randomness associated with sampling.

Transfer-Based Prediction Layer

Rather than treating behaviors like "viewing" and "buying" as isolated events, EHCF models them as a sequence of related tasks. Inspired by knowledge graph embeddings, the model uses a "transfer scheme" where high-level behaviors are viewed as translations of lower-level ones in a shared vector space.

The EHCF Model Framework (a) utilizes a shared embedding layer and a transfer-based prediction layer. The transfer scheme (c) models the relationships between behaviors (b) as translations, allowing knowledge to flow from frequent behaviors like views to rare ones like purchases.

This architecture allows the model to use abundant auxiliary data (e.g., views) to bolster the prediction of target behaviors (e.g., purchases), which are often much sparser.

Experimental Results

Accuracy and Convergence

EHCF was evaluated against state-of-the-art models including Neural Collaborative Filtering (NCF) and Neural Multi-Task Recommendation (NMTR). On the Taobao and Beibei datasets, EHCF showed average improvements of 57.1% and 47.5% respectively over the best baselines.

Convergence analysis on the Taobao dataset demonstrates that EHCF not only reaches a significantly higher Hit Ratio (HR@100) but also converges much faster than competing models.

As illustrated in the figure above, EHCF reaches its peak performance rapidly. In practical terms, on the Beibei dataset, EHCF converged in just 24 minutes, while the previous state-of-the-art (NMTR) required 550 minutes—a 23x improvement in training efficiency.

Robustness to Sparsity

A key strength of EHCF is its performance for "cold-start" users. By effectively transferring patterns from a user's viewing history, the model can make accurate purchase recommendations even for users with very few historical transactions. This robustness is a direct result of the joint optimization of heterogeneous behaviors.

Conclusion

EHCF provides a robust and efficient alternative to traditional negative sampling-based recommendation models. By combining whole-data optimization with a behavior-aware transfer scheme, it sets a new benchmark for both accuracy and speed in heterogeneous collaborative filtering. Future work will focus on extending this non-sampling optimization to non-linear deep neural network architectures.

Summary updated

Abstract

Recent studies on recommendation have largely focused on exploring state-of-the-art neural networks to improve the expressiveness of models, while typically apply the Negative Sampling (NS) strategy for efficient learning. Despite effectiveness, two important issues have not been well-considered in existing methods: 1) NS suffers from dramatic fluctuation, making sampling-based methods difficult to achieve the optimal ranking performance in practical applications; 2) although heterogeneous feedback (e.g., view, click, and purchase) is widespread in many online systems, most existing methods leverage only one primary type of user feedback such as purchase. In this work, we propose a novel non-sampling transfer learning solution, named Efficient Heterogeneous Collaborative Filtering (EHCF) for Top-N recommendation. It can not only model fine-grained user-item relations, but also efficiently learn model parameters from the whole heterogeneous data (including all unlabeled data) with a rather low time complexity. Extensive experiments on three real-world datasets show that EHCF significantly outperforms state-of-the-art recommendation methods in both traditional (single-behavior) and heterogeneous scenarios. Moreover, EHCF shows significant improvements in training efficiency, making it more applicable to real-world large-scale systems. Our implementation has been released 1 to facilitate further developments on efficient whole-data based neural methods.
BibTeX
@inproceedings{chen2020efficient,
  author = {Chong Chen and Min Zhang and Yongfeng Zhang and Weizhi Ma and Yiqun Liu and Shaoping Ma},
  title = {{Efficient Heterogeneous Collaborative Filtering without Negative Sampling for Recommendation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i01.5329},
}
Figure