Attentive Knowledge Graph Embedding for Personalized Recommendation

Xiao Sha, Zhu Sun, Jie ZhangarXiv 2019 · OpenReview

Introduction

Personalized recommendation systems increasingly rely on Knowledge Graphs (KGs) to mitigate the "cold start" problem. By leveraging networks of entities—such as users, items, and their attributes—models can infer preferences even for new users. However, existing approaches often struggle with either oversimplification or excessive noise. This paper introduces the Hierarchical Attentive Knowledge Graph Embedding (HAKG) framework, which focuses on specific subgraphs to capture rich, non-linear relationships while filtering out irrelevant global graph data.

The Problem: Paths vs. Propagation

Current KG-based recommendation methods generally fall into two categories, both with significant drawbacks:

  1. Path-based methods: These models analyze linear chains of relations (e.g., User $\rightarrow$ Friend $\rightarrow$ Item). While interpretable, they fail to capture the complex, "web-like" topology inherent in real-world data.
  2. Propagation-based methods: These models spread information across the entire graph. Although expressive, they often incorporate "noise" from distant, irrelevant nodes, degrading recommendation quality.

HAKG addresses these issues by constructing focused subgraphs for each user-item pair, striking a balance between local complexity and global relevance.

The HAKG Framework

The HAKG framework operates in three primary stages to transform raw graph data into actionable recommendations:

1. Subgraph Construction

For every user-item pair, HAKG samples multiple paths (up to length 6) to build a localized subgraph. This approach ensures the model focuses on the most relevant structural context rather than the entire massive KG.

2. Hierarchical Attentive Subgraph Encoding

This core innovation operates in two steps:

  • Entity Embedding Learning: Utilizing layer-wise propagation, the model updates entity representations based on their neighbors. It specifically incorporates entity and relation types (e.g., "Friendship" vs. "Location") to maintain semantic clarity.
  • Subgraph Embedding Learning: A multi-head self-attention mechanism evaluates all entities within the subgraph to identify those most critical to the user's specific preference.

3. Preference Prediction

The final recommendation score is calculated by feeding the concatenated user, item, and subgraph embeddings into a Multi-Layer Perceptron (MLP).

Key Insights: The Power of Subgraphs

A central finding of this research is that subgraphs provide a significantly stronger signal than isolated linear paths. As illustrated in Figure 1, a user like Mike may be linked to a restaurant like KFC through a variety of overlapping relations, including mutual friends, shared locations (SunCity), and common categories (Food).

Figure 1: A toy example of a Knowledge Graph in the business domain (Yelp), illustrating complex connections between users and items.

By treating these connections as a single, integrated context, HAKG avoids the "irrelevant neighbor" problem. For instance, if a user is searching for food, their connection to a distant hardware store via a social link is effectively filtered out, reducing noise and improving accuracy. The paper demonstrates that being "type-aware"—distinguishing between different kinds of links—is crucial for this process.

Experimental Results

HAKG was evaluated on three diverse real-world datasets: MovieLens-1M, Last-FM, and Yelp. Key findings include:

  • Performance: HAKG consistently outperformed state-of-the-art baselines like KGAT and KPRN, achieving an average relative improvement of 10.7% in Hit@10 and 10.1% in NDCG@10.
  • Sparsity Resilience: The model excels with "inactive users," demonstrating that KG subgraphs can effectively compensate for a lack of historical interaction data.
  • Efficiency: HAKG's sampling strategy resulted in a 76.3% reduction in training time compared to traditional path-based methods like KPRN.

Conclusion

The HAKG framework demonstrates that hierarchical attention over localized subgraphs is a highly effective strategy for KG-based recommendation. By focusing on relevant local structures and remaining aware of entity types, HAKG provides more accurate and efficient recommendations than global propagation or simple path-based models. Future work will explore temporal dynamics and methods to enhance recommendation diversity.

Summary updated

BibTeX
@misc{sha2019attentive,
  author = {Xiao Sha and Zhu Sun and Jie Zhang},
  title = {{Attentive Knowledge Graph Embedding for Personalized Recommendation}},
  year = {2019},
  url = {https://openreview.net/forum?id=AFIaEvJ8fOb},
}
Figure