Improving one-class collaborative filtering by incorporating rich user information

Yanen Li, Jia Hu, ChengXiang Zhai, Ye ChenCIKM 2010 · DOI

TL;DR

This paper tackles the challenge of improving one-class collaborative filtering (OCCF) by incorporating rich, often-overlooked user information like search queries and browsing history. They propose two main strategies: treating this information as independent evidence to be combined with traditional CF scores, or embedding it directly into CF models, showing significant improvements in recommendation accuracy.

What Problem Does This Paper Address?

Traditional collaborative filtering (CF) methods often rely on explicit user ratings (e.g., 1-5 stars). However, in many real-world scenarios, only implicit feedback (e.g., purchases, clicks) is available, making it difficult to gather explicit ratings. This is the domain of One-Class Collaborative Filtering (OCCF), where we only observe positive interactions. OCCF is particularly challenging because the data is extremely sparse and unbalanced. While existing OCCF methods focus on modeling missing data, they largely ignore the wealth of rich user information available in interactive systems, such as search logs, purchase histories, and browsing activities. This paper aims to bridge this gap by demonstrating how to effectively leverage this rich user information to enhance OCCF performance.

Methods

The authors propose two primary strategies for incorporating rich user information into OCCF models:

  1. User Information as Independent Evidence: This approach treats user information (search queries, browsing history) as separate sources of evidence. Scores derived from these sources are then linearly combined with the scores from a standard collaborative filtering method (like User-based CF or Matrix Factorization) to produce a final recommendation score. This allows for a flexible combination of different signals.
  2. Embedding User Information into Collaborative Filtering: This strategy integrates user information directly into the CF model itself.
  3. For User-based CF (UCF), they propose replacing the standard user-user similarity function with a content-based similarity derived from user profiles (e.g., search keywords). They also suggest using denser clickthrough data matrices instead of sparse transactional matrices.
  4. For Matrix Factorization (MF) models (like AMAN and wAMAN), they propose modifying the loss function to incorporate user similarity derived from content features. This influences how missing (negative) examples are weighted, giving less weight to negative examples where the user and item are more similar based on their profiles.

Datasets

The experiments were conducted on a large-scale dataset from a major e-commerce company, specifically focusing on the "CELLPHONE & ACCESSORIES" catalog. The dataset spans 9 weeks and includes:

  • Transactions (purchases)
  • User search logs
  • Clickthrough records

The data was split into training (7 weeks), validation (1 week), and testing (1 week) sets. To handle the long-tail nature of e-commerce items, they employed a generative clustering method to map ephemeral items to more persistent latent product concepts, resulting in 20,409 unique items.

Key Insights and Ideas

  • Rich user information is crucial for OCCF: The paper strongly advocates that leveraging user-specific data beyond simple interaction history is key to overcoming the inherent sparsity and limitations of OCCF.
  • Complementarity is key: The study highlights that user information (like search keywords and clickthroughs) acts as a valuable complement to traditional CF methods, especially when the CF model has weak neighbor support (i.e., for users or items with very few interactions). In such "cold-start" scenarios, content-based signals can significantly boost performance.
  • Search keywords are powerful: Search query history proved to be a more effective source of information than clickthrough data alone, likely because search queries more directly reflect a user's intent and evolving interests.
  • Short-term history is often sufficient: Surprisingly, the experiments suggest that using recent search history (e.g., the last week) can yield performance comparable to, or even better than, using a much longer history. This implies that models can be more efficient by focusing on recent user behavior.
  • MF models can struggle with sparsity: While MF models are powerful, the paper notes that simply treating all missing data as negative (AMAN) or using global weighting (wAMAN) might not perform as well as UCF-based methods on highly sparse OCCF datasets, suggesting that the effectiveness of MF can be dataset-dependent.

Key Results

  • Significant MAP and MPR gains: The proposed methods consistently outperformed baseline OCCF models. For example, combining UCF with search keywords and clickthrough data (UCF+SchKW+CT) achieved a MAP gain of 0.0281 and an MPR gain of 8.2% over the baseline UCF.
  • Embedding user info is effective: Replacing the sparse transactional matrix with a denser clickthrough matrix in UCF (UCFWithCT) improved MAP by 0.0343 and MPR by 10.4% over the UCF baseline, highlighting the impact of data density.
  • Search keywords outperform clickthroughs: In most scenarios, incorporating search keywords led to greater improvements than incorporating clickthrough data alone.
  • Combined information yields best results: The best performance was generally achieved when combining multiple sources of rich user information (e.g., UCF+SchKW+CT and wAMAN+SchKW+CT).
  • Low neighbor support scenarios benefit most: The additive models (UCF+SchKW, UCFWithCT+SchKW+CT) showed the most significant gains over UCF in groups with low neighbor support, demonstrating their ability to mitigate cold-start issues.

Limitations and Future Work

  • Dataset limitations: The authors acknowledge that the MAP scores, even with their proposed methods, are still relatively low compared to other recommendation benchmarks, indicating the inherent difficulty of recommending items in a highly sparse, long-tail e-commerce environment.
  • Model complexity: While effective, some of the proposed methods involve combining multiple models or complex weighting schemes, which can increase computational complexity.
  • Future directions: The authors suggest exploring more sophisticated probabilistic models for unifying various information sources. They also aim to design more efficient and scalable algorithms for combining rich user information, particularly for large-scale industrial applications. Investigating the temporal dynamics of user information further is also a promising avenue.

Summary updated

BibTeX
@inproceedings{li2010improving,
  author = {Yanen Li and Jia Hu and ChengXiang Zhai and Ye Chen},
  title = {{Improving one-class collaborative filtering by incorporating rich user information}},
  booktitle = {ACM International Conference on Information and Knowledge Management},
  year = {2010},
  doi = {10.1145/1871437.1871559},
}
Figure