KQGC: Knowledge Graph Embedding with Smoothing Effects of Graph Convolutions for Recommendation

Daisuke Kikuta, Toyotaro Suzumura, Md. Mostafizur Rahman, Yu Hirate, Satyen Abrol, Manoj Kondapaka, Takuma Ebisu, Pablo LoyolaarXiv 2022 · DOI · arXiv

Introduction

In modern recommendation systems, Knowledge Graphs (KGs) serve as a foundational structure for modeling complex interactions between users, items, and auxiliary attributes such as brands or categories. While translation-based Knowledge Graph Embeddings (KGE), such as TransE, effectively map these entities into a continuous vector space, they often struggle with "structural smoothness." This property ensures that nodes with similar local graph neighborhoods have similar vector representations, which is crucial for robust recommendation performance.

Hybrid models like KGCN and KGAT attempt to address this by integrating Graph Neural Networks (GNNs). However, these models often ignore the specific semantics of relations during aggregation, use complex non-linearities that can disrupt the geometric properties of KGEs, and suffer from high computational costs due to tightly coupled training. To overcome these challenges, we introduce Knowledge Query-based Graph Convolution (KQGC), a modular framework that applies linear graph convolutions to "knowledge queries" to achieve superior structural smoothing.

The KQGC Framework

The KQGC framework adopts a two-stage, modular architecture designed for industrial scalability and geometric consistency.

The KQGC framework architecture, featuring a modular pre-training and smoothing pipeline.
  1. Pre-training (KGE): The process begins by training a standard TransE model on the KG. TransE operates on the principle that the embedding of a head entity ($e_h$) plus the embedding of a relation ($e_r$) should approximately equal the embedding of the tail entity ($e_t$), i.e., $e_h + e_r \approx e_t$.
  2. Smoothing (KQGC): Once the embeddings are pre-trained, they are passed through a Knowledge Query-based Graph Convolution layer. Unlike standard GNNs that aggregate neighbor entity embeddings directly, KQGC aggregates "knowledge queries."

Knowledge Query-based Graph Convolution

The core innovation of KQGC lies in how it defines the information being aggregated. In a translation-based vector space, the sum of a source node and its connecting relation—defined as a Knowledge Query ($Q = e_{source} + e_{relation}$)—acts as a predicted coordinate for where the target node should be located.

Comparison between standard node feature aggregation (a) and the proposed knowledge query aggregation (b).

As shown in the figure above, standard graph convolutions (a) aggregate raw node features, which can lead to noisy representations if the relations are diverse. In contrast, KQGC (b) aggregates queries that point directly toward the expected location of the target node in the TransE space. By using a linear convolution (removing non-linear activations like ReLU), the model preserves the translation properties of the original KGE while effectively "nudging" nodes toward their rightful neighbors.

The authors evaluated three aggregation strategies:

  • Mean Aggregator: A simple uniform average of all incoming knowledge queries.
  • Attention1: A similarity-based aggregator using inner products.
  • Attention2: A more complex aggregator utilizing a trainable MLP-based attention mechanism.

Experimental Evaluation

The model was rigorously tested using real-world data from Rakuten Ichiba, a global e-commerce leader. The dataset comprised approximately 85,000 users and 1.3 million items, connected by 2.3 million edges (primarily purchase relations).

The primary task was Target Prospecting (TP), where the goal is to identify "lookalike" audiences for specific brands. Performance was measured using the Precision-Recall Area Under Curve (PR-AUC).

Key Results

  • Performance Gains: KQGC significantly outperformed both the production baseline (XGBoost with manual features) and the raw TransE model.
  • The Efficacy of Simplicity: Surprisingly, the KQGC(mean) aggregator emerged as the top performer, achieving an average PR-AUC of 0.711. This represents a 3.15% improvement over the baseline.
  • Stability: While attention mechanisms are often preferred in research, the simple mean aggregator proved more stable in the high-homophily environment of e-commerce graphs, suggesting that uniform smoothing is a highly robust strategy for recommendation.

Conclusion and Future Work

KQGC demonstrates that the geometric alignment of users and items can be significantly improved by treating graph convolution as a smoothing operation over translation-based knowledge queries. Its modular design allows for easy integration into existing industrial pipelines, as the KGE component can be pre-trained independently.

Future research will explore the impact of deeper convolutional architectures to determine the limits of the smoothing effect and investigate the inclusion of multimodal side information, such as product images and text descriptions, to further enrich the knowledge graph representations.

Summary updated

Abstract

Leveraging graphs on recommender systems has gained popularity with the development of graph representation learning (GRL). In particular, knowledge graph embedding (KGE) and graph neural networks (GNNs) are representative GRL approaches, which have achieved the state-of-the-art performance on several recommendation tasks. Furthermore, combination of KGE and GNNs (KG-GNNs) has been explored and found effective in many academic literatures. One of the main characteristics of GNNs is their ability to retain structural properties among neighbors in the resulting dense representation, which is usually coined as smoothing. The smoothing is specially desired in the presence of homophilic graphs, such as the ones we find on recommender systems. In this paper, we propose a new model for recommender systems named Knowledge Query-based Graph Convolution (KQGC). In contrast to exisiting KG-GNNs, KQGC focuses on the smoothing, and leverages a simple linear graph convolution for smoothing KGE. A pre-trained KGE is fed into KQGC, and it is smoothed by aggregating neighbor knowledge queries, which allow entity-embeddings to be aligned on appropriate vector points for smoothing KGE effectively. We apply the proposed KQGC to a recommendation task that aims prospective users for specific products. Extensive experiments on a real E-commerce dataset demonstrate the effectiveness of KQGC.
BibTeX
@misc{kikuta2022kqgc,
  author = {Daisuke Kikuta and Toyotaro Suzumura and Md. Mostafizur Rahman and Yu Hirate and Satyen Abrol and Manoj Kondapaka and Takuma Ebisu and Pablo Loyola},
  title = {{KQGC: Knowledge Graph Embedding with Smoothing Effects of Graph Convolutions for Recommendation}},
  year = {2022},
  eprint = {2205.12102},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.12102},
}
Figure