NeuRec: On Nonlinear Transformation for Personalized Ranking

Shuai Zhang, Lina Yao, Aixin Sun, Sen Wang, Guodong Long, Manqing Dong

TL;DR

NeuRec moves beyond the traditional linear assumptions of collaborative filtering by using deep neural networks to map user-item interactions into a non-linear latent space. By integrating multi-layer perceptrons with latent factor models, it achieves state-of-the-art ranking performance while significantly reducing the parameter overhead of previous linear methods.

What Problem Does This Paper Address?

Most classic recommender systems, such as Matrix Factorization (MF) or the Sparse Linear Method (SLIM), rely on a fundamental assumption: the relationship between users and items is linear. They typically model interactions using simple inner products or linear regressions. However, real-world human behavior is messy, dynamic, and rarely linear.

Furthermore, state-of-the-art linear models like SLIM suffer from a "parameter explosion" problem. Because SLIM learns a similarity matrix between all items, its parameter count scales at $N \times N$ (where $N$ is the number of items), making it computationally expensive for large-scale systems. The authors of NeuRec aim to capture these complex, non-linear interaction patterns while keeping the model compact and efficient.

Methods

The authors propose NeuRec, a framework that combines the strengths of Latent Factor Models (LFM) and deep neural networks. The core idea is to use a Multi-Layer Perceptron (MLP) to transform sparse interaction vectors into dense, high-level representations.

The paper introduces two specific architectures:

  1. User-based NeuRec (U-NeuRec): Takes a user's entire interaction history vector as input and passes it through an MLP to generate a dense user latent factor. The final recommendation score is the inner product of this generated user factor and a learned item embedding.
  2. Item-based NeuRec (I-NeuRec): The inverse approach, where an item's interaction history (across all users) is fed into an MLP to generate a dense item latent factor, which is then dotted with a learned user embedding.
Figure 1: Illustration of User-based NeuRec (left) and item-based NeuRec(right). Both of them has two parts: a multi-layer perceptron with Xu∗(or X∗i) as input and item (or user) latent factor.

Key Innovations: * Non-linear Integration: Unlike previous works that use neural networks primarily for auxiliary data (like text or images), NeuRec applies them directly to the interaction matrix to capture non-linearity. * Input Dropout: Instead of standard dropout in hidden layers, the authors apply dropout to the input layer. By randomly "hiding" parts of a user's history during training, they prevent the model from simply learning an identity function and force it to generalize better. * Non-uniform Sampling: For the pairwise learning variant, they use a strategy that samples multiple negative items and picks the "hardest" one (the one with the highest rank) to optimize against, improving ranking quality.

Datasets

The model was evaluated on four diverse, real-world datasets commonly used in recommendation research:

  • Movielens HetRec & Movielens 1M: Standard benchmarks for movie recommendations.
  • FilmTrust: A smaller movie sharing and rating dataset.
  • Frappe: A dataset of Android application usage, representing a different domain (apps vs. media).

All datasets were binarized to represent implicit feedback (e.g., "interacted" vs. "not interacted").

Key Insights and Ideas

  • Linearity is a Bottleneck: The authors explicitly tested an "identity" activation function (which makes the network linear). It performed significantly worse than non-linear functions like Sigmoid, proving that the non-linear transformations are doing the heavy lifting.
  • Parameter Efficiency: By using a bottleneck MLP architecture, NeuRec can reduce the number of parameters by up to 10 times compared to SLIM, while still delivering better accuracy.
  • The "Sweet Spot" for Depth: While deep learning often encourages "going deeper," the authors found that performance plateaus or drops after 3 to 7 hidden layers. Over-complex models tended to overfit the sparse interaction data.
  • Input Sensitivity: The success of input dropout suggests that recommendation models are highly sensitive to the specific items in a user's history; forcing the model to work with partial information is a powerful regularizer.

Key Results

NeuRec consistently outperformed strong baselines, including BPRMF, SLIM, and NeuMF, across all tested metrics (Precision, Recall, MAP, MRR, and NDCG).

Figure 2: NDCG Comparison on dataset (a) Movielens HetRec; (b) Movielens 1M; (c) FilmTrust; (d) Frappe.

Quantitative highlights include:

  • Performance Gains: NeuRec showed improvements over the best baselines by 8.61% on Movielens HetRec, 12.29% on Movielens 1M, and 8.93% on Frappe.
  • Ranking Quality: Higher MRR and NDCG scores indicate that NeuRec is particularly good at pushing the items a user actually likes to the very top of the recommended list.

Limitations and Future Work

  • Model Balancing: The authors noted that U-NeuRec and I-NeuRec perform slightly differently depending on the dataset's user-item distribution. They suggest future work should look at ways to automatically balance or ensemble these two perspectives.
  • Side Information: The current model relies purely on interaction data. The authors plan to incorporate auxiliary information, such as user demographics, item descriptions, and temporal context, to further boost performance.
  • Advanced Regularization: While input dropout was effective, the authors suggest exploring other techniques like Batch Normalization to stabilize training in deeper architectures.

Summary updated

Abstract

Modeling user-item interaction patterns is an important task for personalized recommendations. Many recommender systems are based on the assumption that there exists a linear relationship between users and items while neglecting the intricacy and non-linearity of real-life historical interactions. In this paper, we propose a neural network based recommendation model (NeuRec) that untangles the complexity of user-item interactions and establishes an integrated network to combine non-linear transformation with latent factors. We further design two variants of NeuRec: user-based NeuRec and item-based NeuRec, by concentrating on different aspects of the interaction matrix. Extensive experiments on four real-world datasets demonstrated their superior performances on personalized ranking task.
BibTeX
@inproceedings{zhang2018neurec,
  author = {Shuai Zhang and Lina Yao and Aixin Sun and Sen Wang and Guodong Long and Manqing Dong},
  title = {{NeuRec: On Nonlinear Transformation for Personalized Ranking}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/510},
}
Figure