Understanding and Improving Knowledge Graph Embedding for Entity Alignment

Lingbing Guo, Qiang Zhang, Zequn Sun, Mingyang Chen, Wei Hu, Huajun ChenICML 2022 · OpenReview

Knowledge Graphs (KGs) like DBpedia or Wikidata are massive repositories of structured information, yet they are inherently incomplete. To build a more comprehensive web of knowledge, researchers rely on Entity Alignment (EA) to identify when two different KGs are describing the exact same real-world entity.

Recent advancements have introduced NeoEA (Neural-Ontology-driven Entity Alignment), a framework that improves how we merge KGs by moving beyond simple geometric distance. By using adversarial learning to align "neural ontologies"—the underlying distributions of entity embeddings—NeoEA significantly boosts alignment accuracy, particularly for "long-tail" entities that have few neighbors and are typically hard to match.

The Problem with Margin-Based Scoring

Most modern EA methods use embeddings to represent entities in a vector space, operating on the assumption that if two entities are the same, their vectors should be close together. However, these methods usually rely on a "margin-based" score function (like TransE). This creates a "valid area" for an entity based on its neighbors.

Illustration of margin-based score function and valid areas.

This geometric approach introduces a fundamental dilemma regarding the size of the margin used to constrain these entities:

  1. Small Margins: These constrain entities tightly, which is good for precise alignment but makes it nearly impossible to correctly encode entities with many neighbors (the "valid area" effectively disappears).
  2. Large Margins: These allow for complex, high-degree entities to exist comfortably but leave "long-tail" entities (those with very few neighbors) too loosely constrained, leading to poor alignment.
The influence of margin size on different entities.

The NeoEA Approach: Conditional Neural Axioms

To solve this dilemma, the authors propose NeoEA, a framework that decouples entity alignment into three distinct modules: standard entity alignment, KG sampling, and a novel Neural Ontology Alignment module.

The core innovation of NeoEA is the use of Conditional Neural Axioms. Instead of solely trying to force individual entity vectors close together, NeoEA aligns the distributions of the embeddings.

  • Adversarial Learning: The framework uses a domain critic (based on Wasserstein distance) to ensure that the embedding distributions of two KGs are indistinguishable from one another.
  • Conditioning on Relations: Simply aligning the raw, global distribution of all entities is ineffective because optimized embeddings naturally tend toward a uniform distribution to maximize entropy. Instead, NeoEA aligns distributions conditioned on specific relations (e.g., aligning the distribution of "all head entities connected by the 'genre' relation").

This approach effectively captures high-level ontology rules—such as implicitly learning that the domain of a 'birthPlace' relation is a 'Person'—without requiring any explicit schema or type information from the KGs.

Key Insights and Results

The study tested NeoEA on the OpenEA 100K benchmark, which includes cross-lingual and cross-source datasets (like DBpedia to Wikidata, and English to French) across varying graph densities. NeoEA was evaluated as a "plug-in" for four state-of-the-art baselines (BootEA, SEA, RSN, and RDGCN).

  • The Long-Tail Advantage: The most significant breakthrough is that distribution alignment acts as a powerful regularizer for entities with very few neighbors. While traditional methods leave these entities floating loosely in vector space, NeoEA forces them to conform to the broader distributional patterns of their relations. On the SEA baseline, the ranking improvement for long-tail entities was often more than twice as large as the improvement for popular entities.
  • Universal Improvement: NeoEA consistently improved performance across all datasets and all baseline models.
  • Efficiency: While adversarial training introduces a minimax optimization problem that roughly doubles the total training time compared to simple models, it remains significantly faster than complex iterative methods (like BootEA) while achieving superior results.

Limitations and Future Work

While highly effective, the adversarial training procedure in NeoEA is inherently more sensitive and takes longer to converge than standard loss functions. Additionally, NeoEA's objectives might occasionally conflict with the sophisticated bootstrapping procedures of certain baseline models, suggesting a need for more integrated hyperparameter tuning.

Moving forward, researchers plan to explore how to incorporate actual symbolic ontology knowledge—when it is available—to further refine and guide the neural ontology alignment process.

Summary updated

Abstract

Embedding-based entity alignment (EEA) has recently received great attention. Despite significant performance improvement, few efforts have been paid to facilitate understanding of EEA methods. Mos...
BibTeX
@inproceedings{guo2022understanding,
  author = {Lingbing Guo and Qiang Zhang and Zequn Sun and Mingyang Chen and Wei Hu and Huajun Chen},
  title = {{Understanding and Improving Knowledge Graph Embedding for Entity Alignment}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=hNVz-jpw1J},
}
Figure