SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs

Xiao Liu, Haoyun Hong, Xinghao Wang, Zeyi Chen, Evgeny Kharlamov, Yuxiao Dong, Jie Tang

TL;DR

SelfKG is a novel framework that achieves state-of-the-art entity alignment across Knowledge Graphs (KGs) without using a single training label. By shifting the focus from "pulling" known equivalent entities together to "pushing" non-equivalent entities apart using a Relative Similarity Metric, SelfKG matches or outperforms many supervised models that rely on thousands of hand-labeled examples.

What Problem Does This Paper Address?

Knowledge Graphs (KGs) like Wikidata or DBpedia are massive, but they are often incomplete and siloed. Entity Alignment (EA) is the task of identifying when two different entries in different KGs (e.g., "Apple Inc." in one and "Apple" in another) represent the same real-world entity.

Traditionally, this requires "supervision"—human-labeled pairs of identical entities—to train a model. However, labeling is expensive, slow, and often biased. The challenge is: can we align these massive graphs accurately using only the data within the graphs themselves (self-supervision)?

Methods

The authors introduce SelfKG, which relies on three primary technical innovations to bypass the need for labels:

  1. Relative Similarity Metric (RSM): Standard models use a "pull-and-push" strategy: pull labeled positive pairs closer and push random negative pairs away. SelfKG’s theoretical analysis suggests that for EA, pushing negatives away is actually more beneficial than pulling positives. RSM provides a way to optimize the "uniformity" of the embedding space (pushing everything apart) which implicitly forces equivalent entities together if they share similar semantic features.
  2. Self-Negative Sampling: In unsupervised learning, you might accidentally pick the "true match" from the other KG as a negative sample (a "collision"), which ruins the training. SelfKG avoids this by sampling negative entities from the same KG as the anchor entity, ensuring no true matches are accidentally pushed away.
  3. Multiple Negative Queues (MoCo-inspired): To make the "pushing" effective, the model needs to see thousands of negative examples. Encoding these on the fly is computationally heavy. SelfKG adopts the Momentum Contrast (MoCo) approach, maintaining two large queues (one for each KG) to store and reuse previously encoded entity embeddings efficiently.

The model uses LaBSE (a multilingual BERT variant) for initial entity name embeddings and a single-layer Graph Attention Network (GAT) to incorporate 1-hop neighborhood information.

Datasets

The authors evaluated SelfKG on two gold-standard benchmarks:

  • DWY100K: A monolingual dataset containing 100,000 aligned pairs between DBpedia, Wikidata, and YAGO3.
  • DBP15K: A multilingual dataset (Chinese, Japanese, and French to English) containing 15,000 aligned pairs each.

Key Insights and Ideas

  • The "Push" is Mightier than the "Pull": The most striking insight is that you don't necessarily need to tell a model which entities are the same. If you have a good initial semantic space (from a language model) and you aggressively push non-identical entities apart, the entities that should be together will naturally cluster.
  • 1-Hop is Enough: While many researchers try to use multi-hop graph structures, the authors found that multi-hop information actually harmed performance in a self-supervised setting by introducing too much noise from distant neighbors.
  • Collision Mitigation: By sampling negatives from within the same graph rather than across graphs, the model becomes much more robust to the lack of ground-truth labels.

Key Results

SelfKG achieves remarkable results, often beating supervised models that use 100% of available training labels:

  • DWY100K: SelfKG achieved a Hit@1 of 98.3% to 100%, outperforming almost all supervised baselines except for the very top SOTA (CEAFF).
  • DBP15K: On the multilingual tasks, SelfKG achieved a macro Hit@1 of 84.0% (original) and 89.2% (translated). This significantly beats previous unsupervised methods (like MultiKE at 51.4%) and rivals many supervised GNN-based models.
  • Low-Resource Dominance: When compared to a supervised version of itself in low-data scenarios, SelfKG (with 0% labels) outperformed the supervised version until the latter was given more than 25% of the total labels.

Limitations and Future Work

  • Dependency on Initial Embeddings: SelfKG relies heavily on high-quality initial embeddings from pre-trained language models (like LaBSE). If the entity names are missing or extremely noisy, the "relative similarity" logic may struggle.
  • The Supervision Gap: While SelfKG is close, it still slightly underperforms the absolute best supervised models (like BERT-INT), which can reach nearly 100% accuracy on DBP15K.
  • Future Directions: The authors suggest exploring how to better integrate relation types and more complex graph structures without introducing the noise that currently plagues self-supervised multi-hop approaches.

Summary updated

Abstract

Entity alignment, aiming to identify equivalent entities across different knowledge graphs (KGs), is a fundamental problem for constructing Web-scale KGs. Over the course of its development, the label supervision has been considered necessary for accurate alignments. Inspired by the recent progress of self-supervised learning, we explore the extent to which we can get rid of supervision for entity alignment. Commonly, the label information (positive entity pairs) is used to supervise the process of pulling the aligned entities in each positive pair closer. However, our theoretical analysis suggests that the learning of entity alignment can actually benefit more from pushing unlabeled negative pairs far away from each other than pulling labeled positive pairs close. By leveraging this discovery, we develop the self-supervised learning objective for entity alignment. We present SelfKG with efficient strategies to optimize this objective for aligning entities without label supervision. Extensive experiments on benchmark datasets demonstrate that SelfKG without supervision can match or achieve comparable results with state-of-the-art supervised baselines. The performance of SelfKG suggests that self-supervised learning offers great potential for entity alignment in KGs. The code and data are available at https://github.com/THUDM/SelfKG.
BibTeX
@inproceedings{liu2022selfkg,
  author = {Xiao Liu and Haoyun Hong and Xinghao Wang and Zeyi Chen and Evgeny Kharlamov and Yuxiao Dong and Jie Tang},
  title = {{SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs}},
  booktitle = {The Web Conference},
  year = {2022},
  doi = {10.1145/3485447.3511945},
}

Loading related content...

Figure