A Contextual Alignment Enhanced Cross Graph Attention Network for Cross-lingual Entity Alignment

Zhiwen Xie, Runjie Zhu, Kunsong Zhao, Jin Liu, Guangyou Zhou, Jimmy Xiangji HuangCOLING 2020

TL;DR

This paper introduces CAECGAT, a novel network for aligning entities across different language knowledge graphs. It uniquely leverages "contextual seed alignments" to propagate information between graphs, significantly improving alignment accuracy over existing methods by better capturing semantic relationships.

What Problem Does This Paper Address?

The task of cross-lingual entity alignment is crucial for integrating information from diverse language knowledge graphs (KGs). While many Graph Neural Network (GNN) based methods exist, they often treat KGs independently, learning embeddings separately and only using pre-aligned "seed" entities to optimize the final objective. This approach fails to fully exploit the rich contextual information that these seed alignments provide. For instance, if two entities in different KGs share many aligned neighbors, they are more likely to be equivalent. Existing methods miss this crucial signal, leading to suboptimal alignment performance and difficulty in bridging the semantic gap between languages.

Methods

The core of this paper is the Contextual Alignment Enhanced Cross Graph Attention Network (CAECGAT). It introduces a novel Cross Graph Attention (CGAT) layer designed to jointly learn embeddings across KGs. Each CGAT layer comprises two key components:

  1. Cross-KG Aggregation Layer: This layer uses pre-aligned seed entities to transfer information between KGs. It maps entity embeddings into a shared semantic space by propagating cross-KG information. A "gate mechanism" is employed to control the flow of information between entities and their counterparts in the other KG. Crucially, this layer distinguishes between "contextual seed alignments" (used for information propagation) and "objective seed alignments" (used for training optimization).
  2. Attention-based Cross-KG Propagation Layer: Inspired by Graph Attention Networks (GAT), this layer aggregates neighborhood features. It focuses on important common neighbors shared across KGs, effectively alleviating the semantic gap by prioritizing relevant cross-KG information and downplaying noisy neighbors.

CAECGAT stacks multiple CGAT layers to capture multi-hop cross-KG information. A unique training strategy is also proposed: seed alignments are dynamically split into contextual and objective sets in an iterative manner, ensuring all alignments contribute to both information propagation and model optimization.

Datasets

The experiments were conducted on three widely used cross-lingual datasets extracted from DBP15K:

  • DBP15KZH-EN (Chinese-English)
  • DBP15KJA-EN (Japanese-English)
  • DBP15KFR-EN (French-English)

Each dataset contains 15,000 inter-lingual links.

Key Insights and Ideas

  • Contextual Seed Alignments are Key: The most significant insight is the effective utilization of "contextual seed alignments." By treating a portion of seed alignments not just for optimization but for propagating cross-KG information, the model gains a richer understanding of entity relationships across languages.
  • Bridging the Semantic Gap: The attention mechanism in the propagation layer is crucial for identifying and leveraging shared neighbors, directly addressing the semantic gap between different languages.
  • Multi-hop Information Propagation: Stacking CGAT layers allows the model to capture more complex, multi-hop relationships, leading to more robust entity embeddings.
  • Dynamic Seed Alignment Splitting: The iterative splitting of seed alignments into contextual and objective sets during training is an innovative approach to maximize the utility of available alignment data.

Key Results

CAECGAT achieved state-of-the-art performance on the DBP15K datasets. Specifically:

  • It achieved the best performance across all metrics on DBP15KJA-EN and DBP15KFR-EN.
  • On DBP15KZH-EN, it achieved the best result for H@10 and the second-best for H@1 and MRR.
  • The paper highlights a 34.30% improvement over GCN-Align (a similar GNN-based model) on DBP15KZH-EN for H@1, demonstrating the significant impact of contextual seed alignments.
  • Ablation studies confirmed that both the cross-KG aggregation and attention-based propagation layers are vital for the model's success.

Limitations and Future Work

  • Number of CGAT Layers: While stacking more CGAT layers (e.g., 3 layers) improved performance over 1 layer, the gains diminished with 3 layers compared to 2. The authors suggest that increasing layers further might not yield significant benefits and could increase parameters, thus settling on 2 CGAT layers.
  • Future Work: The authors suggest extending the model to incorporate more entity information, such as entity properties and descriptions. They also propose investigating the model's performance on other datasets.

Summary updated

BibTeX
@inproceedings{xie2020contextual,
  author = {Zhiwen Xie and Runjie Zhu and Kunsong Zhao and Jin Liu and Guangyou Zhou and Jimmy Xiangji Huang},
  title = {{A Contextual Alignment Enhanced Cross Graph Attention Network for Cross-lingual Entity Alignment}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2020},
}
Figure