SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs

Xiao Liu, Haoyun Hong, Xinghao Wang, Zeyi Chen, Evgeny Kharlamov, Yuxiao Dong, Jie Tang

Abstract

Entity alignment (EA) is a fundamental task for constructing Web-scale Knowledge Graphs (KGs) by identifying equivalent entities across different silos. Traditionally, achieving high accuracy in EA has required significant label supervision—human-annotated pairs of identical entities. However, obtaining these labels is expensive and often impractical at scale. Inspired by recent advances in self-supervised learning, we present SelfKG, a framework that achieves state-of-the-art entity alignment without any training labels. By shifting the optimization focus from pulling labeled positive pairs together to pushing unlabeled negative pairs apart via a Relative Similarity Metric (RSM), SelfKG matches or outperforms many supervised models. Our findings suggest that self-supervised learning offers a robust and scalable alternative for KG integration.

Introduction

Knowledge Graphs like Wikidata, DBpedia, and YAGO contain vast amounts of structured information, yet they remain largely disconnected. Entity Alignment (EA) seeks to bridge these gaps. The prevailing paradigm in EA research has relied on supervised learning, where models are trained to minimize the distance between known equivalent entities.

This reliance on supervision presents a bottleneck. Human labeling is slow, prone to bias, and cannot keep pace with the rapid growth of KGs. SelfKG addresses this by asking a fundamental question: can we align entities accurately using only the internal structure and semantic information of the graphs themselves? Our research demonstrates that by leveraging the inherent properties of the embedding space, we can bypass the need for labels entirely.

Methodology

SelfKG introduces three primary technical innovations to enable effective self-supervised alignment:

1. Relative Similarity Metric (RSM)

Standard EA models typically use a "pull-and-push" strategy: they pull labeled positive pairs closer and push random negative pairs away. Our theoretical analysis reveals that for the EA task, the "pushing" of negatives is actually more critical than the "pulling" of positives. RSM optimizes the "uniformity" of the embedding space. By aggressively pushing non-identical entities apart, the model implicitly forces entities with similar semantic and structural features to cluster together, effectively aligning equivalent entities without being explicitly told which ones they are.

2. Self-Negative Sampling

A common pitfall in unsupervised alignment is "collision," where a model accidentally selects a true match from the target KG as a negative sample. This confuses the training process. SelfKG mitigates this by sampling negative entities from the same KG as the anchor entity. This ensures that the model never accidentally pushes away a potential true match from the partner graph.

3. Momentum Contrast (MoCo) Integration

To make the "pushing" mechanism effective, the model must compare an entity against thousands of negative examples. Encoding these on the fly is computationally prohibitive. SelfKG adopts a Momentum Contrast (MoCo) approach, maintaining large, slowly-evolving queues of entity embeddings for each KG. This allows the model to efficiently access a vast pool of negative samples during every training step.

Architecture

The model utilizes LaBSE (Language-agnostic BERT Sentence Embedding) for initial entity name embeddings, capturing cross-lingual semantics. It then employs a single-layer Graph Attention Network (GAT) to incorporate 1-hop neighborhood information. Interestingly, our experiments found that multi-hop information often introduced noise in a self-supervised setting, making the 1-hop approach more robust.

Experimental Evaluation

We evaluated SelfKG on two gold-standard benchmarks: DWY100K (100,000 pairs across DBpedia, Wikidata, and YAGO3) and DBP15K (15,000 pairs across Chinese, Japanese, and French to English).

Key Results

  • Performance on DWY100K: SelfKG achieved a Hit@1 score between 98.3% and 100%. This performance is remarkable as it outperforms nearly all supervised baselines, despite using zero labels.
  • Multilingual Alignment: On the DBP15K dataset, SelfKG achieved a macro Hit@1 of 84.0% (original) and 89.2% (translated). This significantly surpasses previous unsupervised methods like MultiKE (51.4%) and rivals established supervised GNN-based models.
  • Low-Resource Scenarios: In tests against a supervised version of itself, SelfKG (0% labels) consistently outperformed the supervised variant until the latter was provided with more than 25% of the total available labels. This highlights SelfKG's dominance in data-scarce environments.

Discussion and Insights

The success of SelfKG provides a critical insight: the "push" is mightier than the "pull." In high-dimensional embedding spaces, maintaining uniformity by pushing non-identical entities apart is a powerful proxy for alignment. If the initial semantic space (provided by a language model) is sufficiently rich, the structural constraints of the KG are enough to guide the alignment process.

However, SelfKG is not without limitations. It remains dependent on the quality of initial embeddings; if entity names are missing or highly noisy, the relative similarity logic may falter. Furthermore, while SelfKG closes the gap, it still slightly trails the absolute best supervised models (such as BERT-INT) on specific multilingual tasks.

Conclusion

SelfKG demonstrates that label supervision is no longer a prerequisite for high-accuracy entity alignment. By focusing on relative similarity and efficient negative sampling, SelfKG provides a scalable, self-supervised framework for integrating global Knowledge Graphs. Future work will focus on better integrating complex relation types and exploring ways to incorporate multi-hop structures without introducing detrimental noise.

Summary updated

Abstract

Entity alignment, aiming to identify equivalent entities across different knowledge graphs (KGs), is a fundamental problem for constructing Web-scale KGs. Over the course of its development, the label supervision has been considered necessary for accurate alignments. Inspired by the recent progress of self-supervised learning, we explore the extent to which we can get rid of supervision for entity alignment. Commonly, the label information (positive entity pairs) is used to supervise the process of pulling the aligned entities in each positive pair closer. However, our theoretical analysis suggests that the learning of entity alignment can actually benefit more from pushing unlabeled negative pairs far away from each other than pulling labeled positive pairs close. By leveraging this discovery, we develop the self-supervised learning objective for entity alignment. We present SelfKG with efficient strategies to optimize this objective for aligning entities without label supervision. Extensive experiments on benchmark datasets demonstrate that SelfKG without supervision can match or achieve comparable results with state-of-the-art supervised baselines. The performance of SelfKG suggests that self-supervised learning offers great potential for entity alignment in KGs. The code and data are available at https://github.com/THUDM/SelfKG.
BibTeX
@inproceedings{liu2022selfkg,
  author = {Xiao Liu and Haoyun Hong and Xinghao Wang and Zeyi Chen and Evgeny Kharlamov and Yuxiao Dong and Jie Tang},
  title = {{SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs}},
  booktitle = {The Web Conference},
  year = {2022},
  doi = {10.1145/3485447.3511945},
}
Figure