Latent Representation Learning for Autoencoder-based Top-K Recommender System

Dongmin Park, Jun-Hyeok Kang, Jae-Gil LeeJournal of KIISE 2020 · DOI

TL;DR

This paper introduces DenseLR, a novel method to improve autoencoder-based recommender systems by learning more "dense" latent representations. By effectively clustering user or item representations in the latent space, DenseLR significantly enhances collaborative filtering effects, leading to better Top-K recommendations.

What Problem Does This Paper Address?

In today's e-commerce landscape, the sheer volume of products makes it challenging for users to discover items they'll like. Recommender systems are crucial for bridging this gap, satisfying user demand, and boosting sales. While autoencoders have shown promise in collaborative filtering for recommendation tasks, their latent representations often lack optimal structure. Specifically, the distribution of these latent representations isn't fully leveraged to enhance the core collaborative filtering principle: leveraging similarities between users or items. This paper addresses the gap in improving recommendation performance by actively shaping the distribution of latent representations within autoencoder-based models.

Methods

The core contribution is the Dense Latent Representation learning method (DenseLR), which is designed to be integrated with existing autoencoder-based collaborative filtering models. DenseLR comprises two key components:

  • Learnable Normalization Weights: Users or items with more feedback tend to have latent representations with larger norms, pushing them to the "outskirts" of the latent space. This limits effective representation learning. DenseLR introduces learnable weights that are multiplied element-wise with the latent representations. These weights act as a normalization mechanism, allowing the model to adjust the norm of each latent representation, effectively pulling outlier representations closer to the origin and improving their learning.
  • Latent Representation Regularization: This component aims to further enhance the collaborative filtering effect by encouraging latent representations of similar users (or items) to be closer to each other in the latent space. This is achieved by adding a regularization term to the loss function that minimizes the L2 norm of the latent representations. This encourages the model to learn representations that are not only good at reconstructing the input but also capture user-item interaction similarities more effectively.

The authors propose combining these with a pairwise hinge-based loss, which they note is more suitable for Top-K recommendation tasks than point-wise losses.

Datasets

The paper evaluates DenseLR on three widely used real-world datasets for recommender systems:

  • MovieLens 1M: A dataset of movie ratings.
  • Yelp: A dataset of user ratings for businesses (restaurants, etc.).
  • Video Games: A dataset of customer reviews from the Amazon Video Games category.

For MovieLens and Yelp, ratings of 4-5 were treated as positive feedback (1), and ratings of 0-3 as negative feedback (0). For Video Games, purchase history was considered positive feedback. Users and items with fewer than 5 positive feedback instances were removed. The data was split into 80% for training, 10% for validation, and 10% for testing.

Key Insights and Ideas

  • The "outlier" problem: Users with a lot of interaction data (high feedback) often have latent representations that are far from the origin in the latent space. This can hinder effective learning and recommendation.
  • Densifying for better collaboration: By actively pushing similar user/item representations closer together (densifying the latent space), the model can better leverage the "collaborative" aspect of collaborative filtering.
  • Modularity is key: DenseLR is designed to be a plug-and-play module, easily integrated with various autoencoder-based recommender systems, demonstrating its flexibility.
  • Normalization weights matter: The learnable normalization weights are crucial for addressing the outlier problem, allowing for more balanced representation learning.
  • Regularization enhances similarity: The latent representation regularization directly encourages similar users/items to have similar latent representations, which is the core idea of collaborative filtering.

Key Results

The proposed DenseLR method, when combined with the JCA (Joint Collaborative Autoencoders) model, consistently outperformed six state-of-the-art methods across all three datasets and evaluation metrics (F1@k, Precision@k, Recall@k, NDCG@k).

  • Significant performance gains: The combined JCA+DenseLR model achieved the highest F1@k scores on all datasets. The improvements ranged from 4.6% to 23.7% compared to the baseline JCA model.
  • Statistical significance: Statistical tests (p-test) confirmed that the performance improvements were statistically significant.
  • Ablation study: An ablation study showed that both components of DenseLR (learnable normalization weights and latent representation regularization) contribute to performance improvement, with latent representation regularization showing a slightly larger impact.
  • Visualization: t-SNE visualizations demonstrated that the learnable normalization weights effectively pulled outlier representations (from users with high feedback) towards the center of the latent space.
  • Flexibility validated: DenseLR was successfully integrated with other autoencoder-based models (I-AutoRec, U-AutoRec, JCA), showing performance improvements in all cases, highlighting its general applicability.

Limitations and Future Work

  • Over-densification concern: The authors acknowledge that a potential future research direction is to explore methods that don't just pull all latent representations towards a single point (the origin) but rather group them into multiple clusters. This could help differentiate hidden user (or item) types more effectively.
  • Exploring other AE techniques: The paper suggests that further research into other autoencoder-based representation learning techniques, such as modifying prior distributions or changing deep learning architectures for encoders/decoders, could lead to further advancements in autoencoder-based collaborative filtering for recommendations.

Summary updated

Abstract

온라인 상의 상품의 수가 기하 급수적으로 증가함에 따라 고객이 스스로 원하는 상품을 찾는 것이 어려워졌다. 적절한 상품의 추천은 고객의 잠재적 수요를 만족시키고 판매자의 이윤을 증대시키기에 그 중요성이 상당히 크다. 최근에는 인공신경망을 활용한 차원 축소 기법인 오토 인코더 기반의 협업 필터링 방법이 성능 면에서 두각을 나타내었다. 하지만, 오토 인코더의 잠재 표현 분포 조정을 통해 추천 성능을 향상시키는 방법은 아직 많이 연구되지 않았다. 본 연구에서는 오토 인코더 기반 협업 필터링 방법에 결합되어 상품 추천 성능을 더욱 향상시키는 밀집 잠재 표현 학습방법 (DenseLR)을 제안한다. 본 연구의 핵심 아이디어는 유저 구매 정보 벡터들의 잠재 표현을 효과적으로 밀집 시킴에 따라 축소 차원에서의 협업 필터링 효과를 강화하는 것이다. 3가지 실제 구매 데이터 셋에 대해 기존 최첨단 연구들과 성능비교실험을 진행한 결과 제안 방법이 모든 데이터 셋에 대해 가장 높은 성능을 보였다.
BibTeX
@article{park2020latent,
  author = {Dongmin Park and Jun-Hyeok Kang and Jae-Gil Lee},
  title = {{Latent Representation Learning for Autoencoder-based Top-K Recommender System}},
  journal = {Journal of KIISE},
  year = {2020},
  doi = {10.5626/jok.2020.47.2.207},
}
Figure