TransGCN: Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction

Ling Cai, Bo Yan, Gengchen Mai, Krzysztof Janowicz, Rui Zhu

TL;DR

TransGCN is a novel framework that combines the structural awareness of Graph Convolutional Networks (GCNs) with the intuitive logic of transformation-based knowledge graph embeddings (like TransE and RotatE). By treating relations as operators that "homogenize" a node's neighborhood, it learns entity and relation embeddings simultaneously, outperforming previous state-of-the-art models while using significantly fewer parameters.

What Problem Does This Paper Address?

Knowledge Graphs (KGs) are massive networks of real-world facts (triples like subject-relation-object), but they are notoriously incomplete. Predicting missing links is a core challenge in the field.

Existing approaches generally fall into two camps:

  1. Transformation-based models (e.g., TransE, RotatE): These treat relations as geometric transformations (translations or rotations) between entities. They are efficient but often ignore the broader "neighborhood" context of a node.
  2. Graph Convolutional Networks (e.g., R-GCN): These capture local graph structure by aggregating information from neighbors. However, models like R-GCN are "parameter-hungry" because they use separate weight matrices for every relation type and often fail to learn relation embeddings directly within the encoder, requiring a separate decoder for link prediction.

TransGCN aims to bridge this gap by creating a GCN that natively understands relation transformations, making it both structurally aware and parameter-efficient.

Methods

The core innovation of TransGCN is Neighborhood Homogenization.

In a standard KG, a node’s neighbors are connected via different relations (a "heterogeneous" neighborhood), making them hard to aggregate. TransGCN uses a transformation assumption—such as the translation $h + r \approx t$ from TransE or the rotation in complex space from RotatE—to "adjust" the neighbors' embeddings. Essentially, it projects all neighbors into a shared semantic space relative to the center node. Once the neighborhood is homogenized, a standard GCN convolution can be applied.

Key technical features include:

  • Simultaneous Learning: Unlike previous GCNs for KGs, TransGCN updates both entity and relation embeddings at every layer.
  • Relation Updates: Relation embeddings are evolved through layers using a simple linear transformation matrix, ensuring they stay synchronized with the changing entity vector space.
  • Two Variants: The authors instantiate the framework as TransE-GCN (operating in real vector space) and RotatE-GCN (operating in complex vector space).

Datasets

The models were evaluated on two standard, "filtered" benchmarks designed to prevent models from simply memorizing inverse relations:

  • FB15K-237: A subset of Freebase containing 14,541 entities and 237 relations.
  • WN18RR: A subset of WordNet containing 40,943 entities and only 11 relations.

Key Insights and Ideas

  • Structure is a Force Multiplier: Simply wrapping TransE and RotatE in a GCN framework led to noticeable performance gains. This confirms that explicitly modeling local graph structure provides a vital signal that triple-based models miss.
  • Parameter Efficiency: By sharing matrices across relations and using transformation logic, TransGCN avoids the "parameter explosion" seen in R-GCN. This helps alleviate overfitting, especially on smaller datasets.
  • The "Goldilocks" Neighborhood: The authors found that performance increases with entity degree (number of neighbors) up to a point, then drops. Too few neighbors provide insufficient context; too many introduce "mixed information" or noise that makes optimization difficult.
  • Rotation Wins: RotatE-GCN consistently outperformed TransE-GCN. This is likely because the rotation assumption in complex space is mathematically better at capturing complex relational patterns like symmetry, inversion, and composition.

Key Results

TransGCN achieved state-of-the-art results on both datasets:

  • FB15K-237: RotatE-GCN reached a Mean Reciprocal Rank (MRR) of 0.356, significantly higher than the base RotatE (0.338) and R-GCN (0.248).
  • WN18RR: RotatE-GCN achieved an MRR of 0.485, outperforming the base RotatE (0.476).
  • Comparison to R-GCN: TransGCN models showed improvements of 10.8% and 6.7% in MRR over the R-GCN baseline on FB15K-237, while being much more efficient.

Limitations and Future Work

  • Simple Relation Evolution: Currently, relation embeddings are updated via a basic linear transformation. The authors suggest that exploring more complex "convolutions on relations" could further improve the model.
  • Neighbor Weighting: The current model treats all neighbors as equally important. Implementing an attention mechanism (like a Graph Attention Network) to weight neighbor contributions could help filter out noise in very dense neighborhoods.
  • High-Degree Nodes: As noted in their analysis, the model struggles with extremely high-degree nodes. Future work is needed to handle the "information overload" in these dense clusters.

Summary updated

Abstract

Link prediction is an important and frequently studied task that contributes to an understanding of the structure of knowledge graphs (KGs) in statistical relational learning. Inspired by the success of graph convolutional networks (GCN) in modeling graph data, we propose a unified GCN framework, named TransGCN, to address this task, in which relation and entity embeddings are learned simultaneously. To handle heterogeneous relations in KGs, we introduce a novel way of representing heterogeneous neighborhood by introducing transformation assumptions on the relationship between the subject, the relation, and the object of a triple. Specifically, a relation is treated as a transformation operator transforming a head entity to a tail entity. Both translation assumption in TransE and rotation assumption in RotatE are explored in our framework. Additionally, instead of only learning entity embeddings in the convolution-based encoder while learning relation embeddings in the decoder as done by the state-of-art models, e.g., R-GCN, the TransGCN framework trains relation embeddings and entity embeddings simultaneously during the graph convolution operation, thus having fewer parameters compared with R-GCN. Experiments show that our models outperform the-state-of-arts methods on both FB15K-237 and WN18RR.
BibTeX
@inproceedings{cai2019transgcn,
  author = {Ling Cai and Bo Yan and Gengchen Mai and Krzysztof Janowicz and Rui Zhu},
  title = {{TransGCN: Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction}},
  booktitle = {K-CAP},
  year = {2019},
  doi = {10.1145/3360901.3364441},
}

Loading related content...

Figure