Neighborhood-Aware Attentional Representation for Multilingual Knowledge Graphs

Qiannan Zhu, Xiaofei Zhou, Jia Wu, Jianlong Tan, Li GuoIJCAI 2019 · OpenReview

Introduction

Knowledge Graphs (KGs) such as DBpedia, Wikidata, and YAGO represent real-world information through triplets in the form of $(head\_entity, relation, tail\_entity)$. However, these graphs are often language-specific, creating silos of information. Entity alignment is the critical task of identifying equivalent entities across different multilingual KGs (e.g., matching "London" in an English KG to "Londres" in a French KG).

Traditional alignment methods primarily rely on individual triplets to learn embeddings. This paper argues that such a narrow focus ignores the rich structural context provided by an entity's "social circle" or neighborhood. The authors propose NAEA (Neighborhood-Aware Attentional Representation), a model that captures this broader context to improve cross-lingual matching.

Methodology

The NAEA framework consists of two primary components designed to learn robust entity representations for alignment:

  1. Knowledge Embedding (KE) Component: This component learns entity identities using a neighborhood-aware attention mechanism. Unlike previous methods that treat all neighbors equally, NAEA uses self-attention to assign weights to neighbors based on their features and the specific relations connecting them. To ensure training stability and capture diverse structural patterns, the model employs multi-head attention.
  2. Entity Alignment (EA) Component: This component performs the final cross-lingual matching by fusing two distinct types of information: neighbor-level representations (aggregated context from the attention mechanism) and relation-level representations (traditional triplet-based embeddings like TransE).
The NAEA framework architecture, illustrating the flow from neighbor-level attention to the final entity alignment component.

To address the scarcity of pre-aligned entities (the "cold start" problem), NAEA utilizes a bootstrapping strategy, iteratively adding high-confidence alignments to the training set to refine the model.

Experimental Setup

The model was evaluated on two major benchmarks:

  • DBP15K: Three subsets of DBpedia (Chinese-English, Japanese-English, and French-English), each containing 15,000 aligned pairs.
  • DWY100K: Large-scale datasets (100,000 pairs each) extracted from DBpedia, Wikidata, and YAGO3.

Results and Discussion

NAEA consistently outperformed state-of-the-art baselines, including MTransE, JAPE, and BootEA. On the DBP15K (ZH-EN) dataset, NAEA achieved a Hits@1 accuracy of 65.01%, surpassing the previous best of 62.94%. On the larger DWY100K datasets, the model maintained high performance, reaching over 76-77% Hits@1.

The Importance of Information Fusion

A key finding of the study is the optimal balance between neighborhood context and basic triplet structure. The authors introduced a fusion parameter, $\beta$, to weight these two information sources.

Performance (Hit@10) across different datasets as a function of the fusion parameter $\beta$, showing peak performance at $\beta = 0.8$.

As shown in the experimental results, the best performance was achieved at $\beta = 0.8$. This suggests that while neighborhood information is vital for capturing identity, the underlying triplet structure remains a necessary foundation for effective alignment.

Conclusion

The NAEA model demonstrates that looking beyond isolated triplets to an entity's neighborhood significantly enhances multilingual entity alignment. By utilizing multi-head attention to filter informative neighbors and fusing these insights with traditional relation embeddings, NAEA sets a new standard for accuracy and scalability in the field. Future work may involve incorporating additional data types, such as entity descriptions or images, to further refine the attentional weights.

Summary updated

BibTeX
@inproceedings{zhu2019neighborhood,
  author = {Qiannan Zhu and Xiaofei Zhou and Jia Wu and Jianlong Tan and Li Guo},
  title = {{Neighborhood-Aware Attentional Representation for Multilingual Knowledge Graphs}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  url = {https://openreview.net/forum?id=DeZVixUXEU},
}
Figure