TL;DR
SelfKG is a novel framework that achieves state-of-the-art entity alignment across Knowledge Graphs (KGs) without using a single training label. By shifting the focus from "pulling" known equivalent entities together to "pushing" non-equivalent entities apart using a Relative Similarity Metric, SelfKG matches or outperforms many supervised models that rely on thousands of hand-labeled examples.
What Problem Does This Paper Address?
Knowledge Graphs (KGs) like Wikidata or DBpedia are massive, but they are often incomplete and siloed. Entity Alignment (EA) is the task of identifying when two different entries in different KGs (e.g., "Apple Inc." in one and "Apple" in another) represent the same real-world entity.
Traditionally, this requires "supervision"—human-labeled pairs of identical entities—to train a model. However, labeling is expensive, slow, and often biased. The challenge is: can we align these massive graphs accurately using only the data within the graphs themselves (self-supervision)?
Methods
The authors introduce SelfKG, which relies on three primary technical innovations to bypass the need for labels:
- Relative Similarity Metric (RSM): Standard models use a "pull-and-push" strategy: pull labeled positive pairs closer and push random negative pairs away. SelfKG’s theoretical analysis suggests that for EA, pushing negatives away is actually more beneficial than pulling positives. RSM provides a way to optimize the "uniformity" of the embedding space (pushing everything apart) which implicitly forces equivalent entities together if they share similar semantic features.
- Self-Negative Sampling: In unsupervised learning, you might accidentally pick the "true match" from the other KG as a negative sample (a "collision"), which ruins the training. SelfKG avoids this by sampling negative entities from the same KG as the anchor entity, ensuring no true matches are accidentally pushed away.
- Multiple Negative Queues (MoCo-inspired): To make the "pushing" effective, the model needs to see thousands of negative examples. Encoding these on the fly is computationally heavy. SelfKG adopts the Momentum Contrast (MoCo) approach, maintaining two large queues (one for each KG) to store and reuse previously encoded entity embeddings efficiently.
The model uses LaBSE (a multilingual BERT variant) for initial entity name embeddings and a single-layer Graph Attention Network (GAT) to incorporate 1-hop neighborhood information.
Datasets
The authors evaluated SelfKG on two gold-standard benchmarks:
- DWY100K: A monolingual dataset containing 100,000 aligned pairs between DBpedia, Wikidata, and YAGO3.
- DBP15K: A multilingual dataset (Chinese, Japanese, and French to English) containing 15,000 aligned pairs each.
Key Insights and Ideas
- The "Push" is Mightier than the "Pull": The most striking insight is that you don't necessarily need to tell a model which entities are the same. If you have a good initial semantic space (from a language model) and you aggressively push non-identical entities apart, the entities that should be together will naturally cluster.
- 1-Hop is Enough: While many researchers try to use multi-hop graph structures, the authors found that multi-hop information actually harmed performance in a self-supervised setting by introducing too much noise from distant neighbors.
- Collision Mitigation: By sampling negatives from within the same graph rather than across graphs, the model becomes much more robust to the lack of ground-truth labels.
Key Results
SelfKG achieves remarkable results, often beating supervised models that use 100% of available training labels:
- DWY100K: SelfKG achieved a Hit@1 of 98.3% to 100%, outperforming almost all supervised baselines except for the very top SOTA (CEAFF).
- DBP15K: On the multilingual tasks, SelfKG achieved a macro Hit@1 of 84.0% (original) and 89.2% (translated). This significantly beats previous unsupervised methods (like MultiKE at 51.4%) and rivals many supervised GNN-based models.
- Low-Resource Dominance: When compared to a supervised version of itself in low-data scenarios, SelfKG (with 0% labels) outperformed the supervised version until the latter was given more than 25% of the total labels.
Limitations and Future Work
- Dependency on Initial Embeddings: SelfKG relies heavily on high-quality initial embeddings from pre-trained language models (like LaBSE). If the entity names are missing or extremely noisy, the "relative similarity" logic may struggle.
- The Supervision Gap: While SelfKG is close, it still slightly underperforms the absolute best supervised models (like BERT-INT), which can reach nearly 100% accuracy on DBP15K.
- Future Directions: The authors suggest exploring how to better integrate relation types and more complex graph structures without introducing the noise that currently plagues self-supervised multi-hop approaches.
Summary updated