KG-TRICK: Unifying Textual and Relational Information Completion of Knowledge for Multilingual Knowledge Graphs

Zelin Zhou, Simone Conia, Daniel Lee, Min Li, Shenglei Huang, Umar Farooq Minhas, Saloni Potdar, Henry Xiao, Yunyao Li

Abstract

Knowledge Graphs (KGs) serve as the structured backbone for modern artificial intelligence, yet they frequently suffer from incompleteness, particularly in non-English languages. Traditionally, researchers have addressed this through two separate tasks: Knowledge Graph Completion (KGC) for missing relations and Knowledge Graph Enhancement (KGE) for missing textual descriptions. KG-TRICK (Textual and Relational Information Completion of Knowledge) is a novel sequence-to-sequence framework that unifies these tasks into a single multilingual text-to-text generation model. By training on both tasks simultaneously across 10 languages, KG-TRICK achieves state-of-the-art performance, demonstrating that relational and textual data are mutually reinforcing. Notably, the model outperforms massive LLMs like GPT-3.5 on "long-tail" entities, providing a more robust solution for global knowledge representation.

Introduction: The Challenge of Incomplete Knowledge

Knowledge Graphs like Wikidata provide the factual grounding necessary for Large Language Models (LLMs) to operate accurately. However, these graphs are often "patchy." While a KG might contain comprehensive data for a high-profile entity in English, it may lack basic names, aliases, or descriptions in languages such as Thai, Arabic, or Italian.

Historically, the research community has tackled this problem in silos:

  1. Knowledge Graph Completion (KGC): Predicting missing links between entities (e.g., identifying the spouse of a historical figure).
  2. Knowledge Graph Enhancement (KGE): Filling in missing textual metadata (e.g., generating a description for an entity in a specific target language).

KG-TRICK is built on the premise that these tasks are fundamentally interdependent. Relational information provides a language-independent structure that can guide textual descriptions, while textual nuances help disambiguate and identify correct relations.

The KG-TRICK Framework

KG-TRICK reformulates both KGC and KGE as a unified multilingual text-to-text generation task. Instead of using specialized architectures for different data types, the model processes a standardized 5-element tuple: (source_language, target_language, head_entity, relation, ?).

Architecture and Verbalization

The framework utilizes mBART-large-50, a transformer-based encoder-decoder model pre-trained on 50 languages. To bridge the gap between structured data and the model's natural language capabilities, KG triplets are "verbalized" into strings. For example, a KGC query regarding Albert Einstein’s spouse is formatted as: [en] Albert Einstein | spouse | ?.

Multilingual Ensemble

A key innovation of KG-TRICK is its multilingual ensemble module. Because the model is trained across 10 diverse languages, it can generate predictions for the same entity in multiple linguistic contexts. The system then consolidates these outputs into a single, high-confidence entity ID. This process allows the model to effectively "fact-check" itself using cross-lingual information, leading to higher precision in its final outputs.

WikiKGE-10++: A New Benchmark

To evaluate the model, the researchers introduced WikiKGE-10++, a human-verified benchmark for textual completion. This dataset covers over 25,000 entities across 10 languages: Arabic, German, English, Spanish, French, Italian, Japanese, Korean, Thai, and Simplified Chinese.

Crucially, WikiKGE-10++ moves beyond "head" entities (the most popular 10% of data) to include "torso" and "tail" entities. These obscure entities represent the real-world challenge for AI, as they are often missing from standard training sets and are where general-purpose LLMs are most prone to error.

Key Insights and Results

The Synergy Effect

The study confirmed a significant "synergy effect": joint training on KGC and KGE data improves performance on both tasks. Relational data provides the structural skeleton, while textual data adds the necessary linguistic flesh. The authors found a "sweet spot" in training, where mixing 50% KGC data with KGE data yielded the best results, preventing either task from dominating the learning process.

Performance Metrics

  • KGC Superiority: KG-TRICK achieved a Mean Reciprocal Rank (MRR) of 38.8 on the Wikidata5M dataset, outperforming specialized state-of-the-art models like SimKGC and KG-T5.
  • Beating the Giants: On the WikiKGE-10++ benchmark, KG-TRICK significantly outperformed NLLB-200 and Llama3-70B. While GPT-3.5 excels at describing famous entities, its performance drops significantly for "tail" entities. KG-TRICK maintained consistent accuracy across all popularity tiers.
  • Downstream Utility: When used as a pre-training step for multilingual Question Answering (using the Mintaka dataset), KG-TRICK improved mBART's accuracy by 3.1%, proving that the knowledge captured by the model is transferable to real-world reasoning tasks.

Limitations and Future Directions

While KG-TRICK represents a major step forward, it currently operates under a "closed-world assumption," meaning it can only predict entities that already exist within the graph. It cannot yet synthesize entirely new entities from external web sources. Additionally, as a generative model, it carries a risk of hallucination—generating plausible but incorrect descriptions. Future iterations may focus on expanding the variety of textual facts (such as mottos or quotes) and incorporating human-in-the-loop verification for production environments.

Conclusion

KG-TRICK demonstrates that the path to more complete and equitable Knowledge Graphs lies in unification. By breaking down the barriers between relational and textual data and embracing a multilingual approach, the framework provides a scalable way to ground AI in facts across the global linguistic landscape.

Summary updated

BibTeX
@inproceedings{zhou2025kg,
  author = {Zelin Zhou and Simone Conia and Daniel Lee and Min Li and Shenglei Huang and Umar Farooq Minhas and Saloni Potdar and Henry Xiao and Yunyao Li},
  title = {{KG-TRICK: Unifying Textual and Relational Information Completion of Knowledge for Multilingual Knowledge Graphs}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2025},
}
Figure