Abstract
Entity alignment (EA) is a fundamental task for constructing Web-scale Knowledge Graphs (KGs) by identifying equivalent entities across different silos. Traditionally, achieving high accuracy in EA has required significant label supervision—human-annotated pairs of identical entities. However, obtaining these labels is expensive and often impractical at scale. Inspired by recent advances in self-supervised learning, we present SelfKG, a framework that achieves state-of-the-art entity alignment without any training labels. By shifting the optimization focus from pulling labeled positive pairs together to pushing unlabeled negative pairs apart via a Relative Similarity Metric (RSM), SelfKG matches or outperforms many supervised models. Our findings suggest that self-supervised learning offers a robust and scalable alternative for KG integration.
Introduction
Knowledge Graphs like Wikidata, DBpedia, and YAGO contain vast amounts of structured information, yet they remain largely disconnected. Entity Alignment (EA) seeks to bridge these gaps. The prevailing paradigm in EA research has relied on supervised learning, where models are trained to minimize the distance between known equivalent entities.
This reliance on supervision presents a bottleneck. Human labeling is slow, prone to bias, and cannot keep pace with the rapid growth of KGs. SelfKG addresses this by asking a fundamental question: can we align entities accurately using only the internal structure and semantic information of the graphs themselves? Our research demonstrates that by leveraging the inherent properties of the embedding space, we can bypass the need for labels entirely.
Methodology
SelfKG introduces three primary technical innovations to enable effective self-supervised alignment:
1. Relative Similarity Metric (RSM)
Standard EA models typically use a "pull-and-push" strategy: they pull labeled positive pairs closer and push random negative pairs away. Our theoretical analysis reveals that for the EA task, the "pushing" of negatives is actually more critical than the "pulling" of positives. RSM optimizes the "uniformity" of the embedding space. By aggressively pushing non-identical entities apart, the model implicitly forces entities with similar semantic and structural features to cluster together, effectively aligning equivalent entities without being explicitly told which ones they are.
2. Self-Negative Sampling
A common pitfall in unsupervised alignment is "collision," where a model accidentally selects a true match from the target KG as a negative sample. This confuses the training process. SelfKG mitigates this by sampling negative entities from the same KG as the anchor entity. This ensures that the model never accidentally pushes away a potential true match from the partner graph.
3. Momentum Contrast (MoCo) Integration
To make the "pushing" mechanism effective, the model must compare an entity against thousands of negative examples. Encoding these on the fly is computationally prohibitive. SelfKG adopts a Momentum Contrast (MoCo) approach, maintaining large, slowly-evolving queues of entity embeddings for each KG. This allows the model to efficiently access a vast pool of negative samples during every training step.
Architecture
The model utilizes LaBSE (Language-agnostic BERT Sentence Embedding) for initial entity name embeddings, capturing cross-lingual semantics. It then employs a single-layer Graph Attention Network (GAT) to incorporate 1-hop neighborhood information. Interestingly, our experiments found that multi-hop information often introduced noise in a self-supervised setting, making the 1-hop approach more robust.
Experimental Evaluation
We evaluated SelfKG on two gold-standard benchmarks: DWY100K (100,000 pairs across DBpedia, Wikidata, and YAGO3) and DBP15K (15,000 pairs across Chinese, Japanese, and French to English).
Key Results
- Performance on DWY100K: SelfKG achieved a Hit@1 score between 98.3% and 100%. This performance is remarkable as it outperforms nearly all supervised baselines, despite using zero labels.
- Multilingual Alignment: On the DBP15K dataset, SelfKG achieved a macro Hit@1 of 84.0% (original) and 89.2% (translated). This significantly surpasses previous unsupervised methods like MultiKE (51.4%) and rivals established supervised GNN-based models.
- Low-Resource Scenarios: In tests against a supervised version of itself, SelfKG (0% labels) consistently outperformed the supervised variant until the latter was provided with more than 25% of the total available labels. This highlights SelfKG's dominance in data-scarce environments.
Discussion and Insights
The success of SelfKG provides a critical insight: the "push" is mightier than the "pull." In high-dimensional embedding spaces, maintaining uniformity by pushing non-identical entities apart is a powerful proxy for alignment. If the initial semantic space (provided by a language model) is sufficiently rich, the structural constraints of the KG are enough to guide the alignment process.
However, SelfKG is not without limitations. It remains dependent on the quality of initial embeddings; if entity names are missing or highly noisy, the relative similarity logic may falter. Furthermore, while SelfKG closes the gap, it still slightly trails the absolute best supervised models (such as BERT-INT) on specific multilingual tasks.
Conclusion
SelfKG demonstrates that label supervision is no longer a prerequisite for high-accuracy entity alignment. By focusing on relative similarity and efficient negative sampling, SelfKG provides a scalable, self-supervised framework for integrating global Knowledge Graphs. Future work will focus on better integrating complex relation types and exploring ways to incorporate multi-hop structures without introducing detrimental noise.
Summary updated