Hybrid Recommender System based on Autoencoders

Florian Strub, Jérémie Mary, Romaric Gaudel

Introduction

Collaborative Filtering (CF) remains the cornerstone of modern recommendation systems, predicting user preferences by analyzing historical interaction patterns across a broad user base. Traditionally, this is framed as a Matrix Factorization (MF) problem—filling in the missing entries of a sparse user-item rating matrix. However, traditional MF and standard Neural Networks (NNs) struggle with two primary issues: extreme data sparsity (often >95% missing) and the "cold start" problem, where new users or items lack sufficient historical data for accurate prediction.

This paper introduces the Collaborative Filtering Network (CFN), a robust framework based on autoencoders that addresses these challenges through specialized loss functions and deep integration of side information.

The Collaborative Filtering Network (CFN) Architecture

The CFN adapts the autoencoder architecture—traditionally used for data compression and reconstruction—to the specific needs of recommendation tasks. The authors propose two variants: U-CFN (user-based), which processes vectors of all ratings by a single user, and I-CFN (item-based), which processes all ratings given to a specific movie or product.

Handling Sparsity and Corruption

To manage missing data, the CFN inhibits input and output nodes corresponding to unrated items. During training, the network only back-propagates errors for known ratings. Furthermore, the model employs a Denoising Autoencoder (DAE) approach, intentionally "corrupting" a portion of the known ratings by masking them. This forces the network to learn predictive patterns rather than simply memorizing existing data.

The CFN training pipeline: The forward pass (left) handles missing values and corruption, while the backward pass (right) applies a weighted loss function to prioritize reconstruction of masked values.

The Weighted Loss Function

A key innovation is the weighted loss function, $L_{\alpha, \beta}$. By assigning a higher weight ($\alpha$) to predicting masked/missing values and a lower weight ($\beta$) to reconstructing known ratings, the model is incentivized to prioritize the discovery of underlying latent structures. This ensures the network focuses on its actual goal: filling in the gaps of the rating matrix.

Hybrid Integration of Side Information

To overcome the cold start problem, the CFN incorporates "side information"—such as user demographics (age, gender) or item metadata (genres, tags). Unlike traditional methods that simply append this data to the input layer, the CFN "wires" side information into every hidden layer of the network.

Hybrid CFN architecture: Side information is injected into every hidden layer, ensuring it deeply influences the learned representations at all levels of abstraction.

This deep injection ensures that metadata significantly influences the latent features learned by the network, providing a "safety net" for items or users with sparse rating histories. For well-rated items, the ratings themselves provide enough signal, but for the "long tail" of the dataset, this side information is crucial.

Key Insights and Experimental Results

The authors' experiments on major benchmarks like MovieLens (1M, 10M, 20M) and Douban yielded several critical findings:

  • Non-Linear Matrix Factorization: A single-hidden-layer autoencoder acts as a non-linear generalization of Matrix Factorization, where the hidden layer represents latent factors.
  • Item-Based Superiority: I-CFN consistently outperformed U-CFN, suggesting that item-based structures (the profile of users who like a specific movie) are more stable and learnable than the often-shifting tastes of individual users.
  • State-of-the-Art Performance: I-CFN achieved an RMSE of 0.7767 on MovieLens-10M, surpassing previous benchmarks like Autorec. The addition of side information further improved this to 0.7754.
  • Efficiency and Scalability: Leveraging GPU acceleration, the model trained on the 20-million-rating MovieLens dataset in just 34 minutes, demonstrating high practical utility for large-scale commercial systems.

Conclusion

The Collaborative Filtering Network provides a powerful, scalable solution for modern recommendation challenges. By combining denoising techniques with a novel multi-layer injection of side information, it effectively bridges the gap between pure collaborative filtering and content-based recommendation, setting a new standard for accuracy and robustness in the field.

Summary updated

Abstract

A standard model for Recommender Systems is the Matrix Completion setting: given partially known matrix of ratings given by users (rows) to items (columns), infer the unknown ratings. In the last decades, few attempts where done to handle that objective with Neural Networks, but recently an architecture based on Autoencoders proved to be a promising approach. In current paper, we enhanced that architecture (i) by using a loss function adapted to input data with missing values, and (ii) by incorporating side information. The experiments demonstrate that while side information only slightly improve the test error averaged on all users/items, it has more impact on cold users/items.
BibTeX
@inproceedings{strub2016hybrid,
  author = {Florian Strub and Jérémie Mary and Romaric Gaudel},
  title = {{Hybrid Recommender System based on Autoencoders}},
  booktitle = {DLRS@RecSys},
  year = {2016},
  url = {https://openreview.net/forum?id=x_yO-miDW6},
}
Figure