Revisit and Outstrip Entity Alignment: A Perspective of Generative Models

Lingbing Guo, Zhuo Chen, Jiaoyan Chen, Yin Fang, Wen Zhang, Huajun Chen

TL;DR

This paper rethinks Entity Alignment (EA) through the lens of generative models, proving that generative objectives theoretically enhance alignment performance. The authors introduce GEEA, a framework using a Mutual Variational Autoencoder (M-VAE) that not only sets new state-of-the-art records for alignment but also enables "entity synthesis"—the ability to generate entirely new entities with realistic attributes and relationships in a target knowledge graph.

The Challenge of Entity Alignment

In the ecosystem of Knowledge Graphs (KGs), Entity Alignment (EA) is the critical task of identifying pairs of entities from different graphs that represent the same real-world object (e.g., matching "Star Wars" in an English KG to "Guerra de las Galaxias" in a Spanish one).

Historically, most existing methods have been discriminative: they attempt to learn a mapping that minimizes the distance between aligned entity embeddings. However, this approach struggles with "dangling entities"—entities that exist in one KG but have no counterpart in the other. Furthermore, existing generative attempts (mostly GAN-based) suffer from "mode collapse," where the model produces repetitive, unhelpful embeddings, and they lack the ability to translate an entity's actual features (like its neighbors or attributes) from one graph to another.

Introducing GEEA and the M-VAE

The authors propose GEEA (Generative Embedding-based Entity Alignment), which shifts away from simple discriminative matching toward a robust reconstruction-based approach using a Mutual Variational Autoencoder (M-VAE).

The GEEA framework architecture, illustrating the Variational Autoencoder, Prior/Post Reconstruction, and Distribution Matching components.

Key technical components of this framework include:

  • M-VAE Architecture: Unlike a standard VAE, the M-VAE handles four distinct "flows." Two are self-supervised ($x \to x$ and $y \to y$), allowing the model to learn from every entity in the graphs. The other two are supervised ($x \to y$ and $y \to x$), using a small set of known "seed" alignments to learn how to convert entities between graphs.
  • Prior & Post Reconstruction: The model goes beyond generating abstract vectors. It uses decoders to reconstruct "concrete features," such as an entity's neighborhood in the graph or its specific attributes (e.g., birth date, film genre). "Post reconstruction" ensures that these different modalities (image, text, graph) remain consistent for a single synthesized entity.
  • Distribution Matching: To avoid the training instability typical of GANs, GEEA forces the latent variables of both KGs to match a standard normal distribution. The authors provide a formal proof that this alignment in latent space naturally leads to better entity alignment in the output space.

Key Insights and Theoretical Advantages

  • The Generative Advantage: The authors mathematically demonstrate that maximizing the Evidence Lower Bound (ELBO)—the standard objective for VAEs—is theoretically equivalent to minimizing the prediction error in entity alignment. This provides a solid foundation for why generative models are better suited for this task.
  • Stability over Competition: By using a VAE with a fixed Gaussian prior instead of a GAN with a competing discriminator, the model avoids mode collapse. This makes the training much more stable and the resulting embeddings significantly more diverse.
  • Entity Synthesis: This is the "outstrip" aspect of the research. GEEA can take a "dangling" entity from KG A and synthesize what it would look like in KG B, predicting its likely neighbors and attributes. This represents a massive step forward for comprehensive knowledge graph integration.

Experimental Results

The model was rigorously evaluated on several multi-modal benchmarks, including cross-lingual datasets derived from DBpedia (DBP15K) and complex relational datasets (FB15K-DB15K & FB15K-YAGO15K).

Performance comparison of GEEA against baselines, showing entity synthesis metrics (table) and alignment accuracy across different training data ratios (charts).
  • State-of-the-Art Performance: GEEA consistently outperformed existing SOTA methods (like MCLEA and NeoEA) across all datasets. For example, on DBP15K (ZH-EN), it achieved a Hits@1 of 0.761, marking a significant jump over previous benchmarks.
  • Data Efficiency: GEEA is remarkably robust when training data is scarce. When using only 10% of the available seed alignments, GEEA outperformed the next best method by 36.1% in Hits@1.
  • Qualitative Synthesis: In practical tests, GEEA successfully synthesized missing entities. For instance, given "Star Wars" from a source KG, it correctly predicted target KG attributes like imdbId and initial release date, even when those specific fields weren't explicitly present in the source.

Limitations and Future Work

While highly effective, GEEA does introduce computational overhead. Because it includes multiple decoders to reconstruct concrete features, it has a higher parameter count and requires more memory than simpler discriminative models. Additionally, the quality of entity synthesis exhibits feature dependency; if the source KG is sparse, the synthesized entity in the target KG will inherently be limited.

Future directions suggested by the authors include exploring more advanced multi-modal encoders and moving toward models that can process and generate raw data (like actual text or pixels) directly, rather than relying on pre-trained embeddings.

Summary updated

Abstract

Recent embedding-based methods have achieved great successes in exploiting entity alignment from knowledge graph (KG) embeddings of multiple modalities. In this paper, we study embedding-based entity alignment (EEA) from a perspective of generative models. We show that EEA shares similarities with typical generative models and prove the effectiveness of the recently developed generative adversarial network (GAN)-based EEA methods theoretically. We then reveal that their incomplete objective limits the capacity on both entity alignment and entity synthesis (i.e., generating new entities). We mitigate this problem by introducing a generative EEA (GEEA) framework with the proposed mutual variational autoencoder (M-VAE) as the generative model. M-VAE enables entity conversion between KGs and generation of new entities from random noise vectors. We demonstrate the power of GEEA with theoretical analysis and empirical experiments on both entity alignment and entity synthesis tasks. The source code and datasets are available at github.com/zjukg/GEEA.
BibTeX
@inproceedings{guo2024revisit,
  author = {Lingbing Guo and Zhuo Chen and Jiaoyan Chen and Yin Fang and Wen Zhang and Huajun Chen},
  title = {{Revisit and Outstrip Entity Alignment: A Perspective of Generative Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=z3dfuRcGAK},
}
Figure