Auto-Encoding User Ratings via Knowledge Graphs in Recommendation Scenarios

Vito Bellini, Vito Walter Anelli, Tommaso Di Noia, Eugenio Di SciascioarXiv 2017 · DOI · arXiv

TL;DR

The authors introduce SEM-AUTO, a semantic autoencoder that uses Knowledge Graphs (like DBpedia) to define its neural network topology. By mapping hidden units to explicit categories rather than abstract latent factors, the model creates interpretable user profiles that significantly outperform state-of-the-art baselines for "cold-start" users with as few as two ratings.

What Problem Does This Paper Address?

Recommender systems generally fall into two camps: Collaborative Filtering (CF), which looks at what similar users liked, and Content-Based (CB) filtering, which looks at item characteristics. While CF is powerful, it suffers from the cold-start problem: it fails when a user has provided very few ratings because there isn't enough data to find reliable "neighbors."

Deep learning approaches, specifically autoencoders, have been used to bridge this gap, but they typically produce "black box" latent representations. We might know a user likes "Factor 42," but we have no idea what "Factor 42" actually represents. The authors seek to combine the representational power of autoencoders with the interpretability and external knowledge of Knowledge Graphs to solve the cold-start problem.

Methods

The core innovation is the SEM-AUTO architecture, which moves away from the standard fully-connected layers found in traditional neural networks.

  1. Knowledge-Driven Topology: The network's structure is built "on-the-fly" based on a Knowledge Graph (KG). The input and output layers represent items (e.g., movies), while the hidden layer represents categorical features from the KG (e.g., "Western Films" or "American Films").
  2. Sparse Connections: An edge exists between an input item and a hidden category neuron only if the KG explicitly links them. This means the network is not fully connected; its architecture reflects the actual semantic relationships in the data.
  3. Semantic Weighting: As the autoencoder learns to reconstruct user ratings, the weights on these edges represent the importance of specific semantic features to that user. Because each hidden unit corresponds to a real-world category, these weights are directly interpretable.
  4. Hybrid Enhancement: To further boost performance, the model uses a collaborative step. It projects users into a "feature space" based on their learned weights, finds similar users, and uses their data to estimate weights for features the original user hasn't encountered yet.
Figure 1: Architecture of a semantic autoencoder.

Datasets

The researchers evaluated their approach using:

  • MovieLens 1M: A classic benchmark dataset containing ~6,000 users and ~4,000 movies.
  • DBpedia: A crowd-sourced community effort to extract structured information from Wikipedia. It served as the external Knowledge Graph to provide movie categories via dct:subject and rdf:type predicates.
  • Mapping: The authors used a pre-existing mapping to link 3,573 movies from MovieLens to their corresponding DBpedia entries.

Key Insights and Ideas

  • Interpretability by Design: By forcing the hidden layer to correspond to KG categories, the "latent space" is no longer latent—it's explicit. You can look at a user's profile and see exactly how much they value "Romantic Epic films" versus "Action films."
  • The Power of Priors: The KG acts as a massive "prior" for the model. Even with only two ratings, the model knows those movies belong to specific genres, allowing it to make much more educated guesses than a standard CF model that treats items as opaque IDs.
  • Efficiency through Sparsity: Because the network is not fully connected, it has fewer parameters to train, which can help prevent overfitting when data is extremely scarce.

Key Results

The authors focused on "cold" scenarios where users had only 2, 5, or 10 ratings.

  • Dominance in Extreme Cold-Start: For users with only 2 ratings, SEM-AUTO significantly outperformed the state-of-the-art BPRSLIM baseline across all metrics, including Precision, nDCG (ranking quality), and ERR-IA (diversity).
  • Diminishing Returns: As the number of ratings increases, the advantage of the semantic approach narrows. By 10 ratings, the collaborative baseline (BPRSLIM) begins to overtake SEM-AUTO, as there is enough internal data for pure CF to work effectively.
Figure 4: Plot for 2 ratings.

Detailed results in the paper's table show that for 2-rating users, SEM-AUTO achieved an nDCG@10 of 0.028 compared to BPRSLIM's 0.023, a notable improvement in ranking accuracy for such a data-poor scenario.

Limitations and Future Work

  • KG Coverage: The model is limited by the quality and coverage of the Knowledge Graph. If a movie isn't in DBpedia or lacks categorical tags, SEM-AUTO cannot process it.
  • Computational Overhead: Building and training custom network topologies for users can be more computationally intensive than traditional matrix factorization methods.
  • Future Directions: The authors plan to test the model on other KGs like Wikidata and explore using these semantic features to generate natural language explanations for recommendations (e.g., "We recommend this because you have a high preference for 1960s Italian cinema").

Summary updated

Abstract

In the last decade, driven also by the availability of an unprecedented computational power and storage capabilities in cloud environments we assisted to the proliferation of new algorithms, methods, and approaches in two areas of artificial intelligence: knowledge representation and machine learning. On the one side, the generation of a high rate of structured data on the Web led to the creation and publication of the so-called knowledge graphs. On the other side, deep learning emerged as one of the most promising approaches in the generation and training of models that can be applied to a wide variety of application fields. More recently, autoencoders have proven their strength in various scenarios, playing a fundamental role in unsupervised learning. In this paper, we instigate how to exploit the semantic information encoded in a knowledge graph to build connections between units in a Neural Network, thus leading to a new method, SEM-AUTO, to extract and weigh semantic features that can eventually be used to build a recommender system. As adding content-based side information may mitigate the cold user problems, we tested how our approach behave in the presence of a few rating from a user on the Movielens 1M dataset and compare results with BPRSLIM.
BibTeX
@misc{bellini2017auto,
  author = {Vito Bellini and Vito Walter Anelli and Tommaso Di Noia and Eugenio Di Sciascio},
  title = {{Auto-Encoding User Ratings via Knowledge Graphs in Recommendation Scenarios}},
  year = {2017},
  eprint = {1706.07956},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1706.07956},
}

Loading related content...

Figure