VECTORIZATION METHODS IN RECOMMENDER SYSTEM

Qiang Sun, Bin Wang, Zizhou Gu, Yanwei FuOpenReview 2018 · OpenReview

TL;DR

This paper explores using natural language processing (NLP) techniques, specifically word2vec and doc2vec, to improve recommender systems. They propose two novel vectorization methods: user2vec for representing users and doc2vec for representing items based on their descriptions, aiming to enhance recommendation accuracy and address the cold-start problem.

What Problem Does This Paper Address?

Recommender systems are crucial for helping users discover relevant items, but they face challenges like the "cold-start problem" where new users or items lack sufficient interaction data for traditional collaborative filtering methods. While collaborative filtering relies on user-item interaction patterns, it can miss valuable information embedded in item descriptions or user behavior nuances. This paper aims to leverage advancements in NLP, particularly word embeddings, to create richer representations of users and items, thereby improving recommendation quality.

Methods

The paper introduces two main vectorization methods:

  • user2vec: This is presented as a mirror of the item2vec model. It treats user IDs as "words" and constructs "sentences" based on users who have rated the same item similarly (e.g., users who gave a 4-5 star rating form a "like sentence," and 1-3 stars form a "dislike sentence"). The order of user IDs within these sentences is shuffled before training. This approach allows users to be represented as vectors, enabling user-based collaborative filtering through cosine similarity.
  • doc2vec: This method utilizes the doc2vec technique to convert item descriptions into dense vectors. The idea is that items with similar descriptions will have similar vector representations. These item vectors are then used to compute item-item similarity, which is subsequently employed in an item-based collaborative filtering framework.

Both methods build upon the success of word embedding models like word2vec and doc2vec from NLP.

Datasets

The experiments were conducted on a subset of the Amazon dataset for video games. This dataset originally contained 231K reviews and item metadata including user ID, item ID, rating, and item descriptions. After preprocessing (removing items without descriptions and users with fewer than 5 interactions), the dataset comprised 146K reviews, 15K users, and 9K items. This data was split into training (60%), validation (20%), and testing (20%) sets.

Key Insights and Ideas

  • User and Item Vectorization is Powerful: The paper demonstrates that treating users and items as entities that can be vectorized, similar to words in NLP, can significantly enhance recommender systems.
  • user2vec as a Complement to item2vec: By mirroring item2vec, user2vec offers a way to capture user similarities directly from interaction patterns, potentially uncovering different types of relationships than item-based approaches.
  • Leveraging Textual Descriptions with doc2vec: Item descriptions, often rich in semantic information, can be effectively converted into vectors using doc2vec. This allows for content-based similarity calculations that can help mitigate the cold-start problem by providing a basis for recommending new items.
  • Item Frequency Matters: The authors note that item2vec outperformed user2vec partly because item frequencies in reviews are generally higher than user frequencies, which benefits the training of embedding models.

Key Results

The paper compares their proposed methods (user2vec, doc2vec) against a baseline knn-cf (k-nearest neighbors collaborative filtering) and item2vec.

  • item2vec achieved the best performance on the test set.
  • user2vec performed slightly better than the knn-cf baseline.
  • doc2vec also outperformed the knn-cf baseline.

The authors attribute the success of doc2vec to both the effectiveness of the model and the descriptive nature of video game text, which reflects item attributes.

Method RMSE MAE
knn-cf 1.145 0.839
item2vec 1.135 0.849
user2vec 1.142 0.859
doc2vec 1.142 0.855

Limitations and Future Work

  • User vs. Item Frequency: The performance difference between item2vec and user2vec highlights the impact of data sparsity and frequency on embedding models.
  • Future Work: The authors plan to explore the relationship between the user vector space and the item vector space further. They also aim to investigate how to combine these vector representations more effectively.

Summary updated

Abstract

The most used recommendation method is collaborative filtering, and the key part of collaborative filtering is to compute the similarity. The similarity based on co-occurrence of similar event is easy to implement and can be applied to almost all the situation. So when the word2vec model reach the state-of-art at a lower computation cost in NLP. An correspond model in recommender system item2vec is proposed and reach state-of-art in recommender system. It is easy to see that the position of user and item is interchangeable when their count size gap is not too much, we proposed a user2vec model and show its performance. The similarity based on co-occurrence information suffers from cold start, we proposed a content based similarity model based on doc2vec which is another technology in NLP.
BibTeX
@misc{sun2018vectorization,
  author = {Qiang Sun and Bin Wang and Zizhou Gu and Yanwei Fu},
  title = {{VECTORIZATION METHODS IN RECOMMENDER SYSTEM}},
  year = {2018},
  url = {https://openreview.net/forum?id=HylJtiRqYQ},
  howpublished = {OpenReview},
}
Figure