Abstract
Neural Machine Translation (NMT) systems typically require vast amounts of parallel training data to achieve high performance. However, they frequently struggle with "entities"—specific names or technical terms—that are absent from the training corpus ($K-D$ entities). This paper proposes a three-step pipeline to leverage structured Knowledge Graphs (KGs) to bridge this gap. By aligning bilingual KGs into a unified semantic space and generating synthetic "pseudo-parallel" training data, the authors enable NMT models to accurately translate rare and unseen entities. Experimental results on Chinese-to-English and English-to-Japanese tasks demonstrate significant improvements in BLEU scores, particularly for sentences containing rare entities.
Introduction
Standard NMT models learn translation mappings by observing millions of sentence pairs. While effective for common vocabulary, this data-driven approach fails when encountering rare entities or domain-specific terminology not represented in the parallel data. While Knowledge Graphs like DBpedia contain rich information about such entities, existing research often focuses on entities already present in the training data ($K+D$ entities).
This work addresses the more challenging problem of $K-D$ entities: those found in KGs but missing from the parallel training set. Without external knowledge, NMT models often "hallucinate" translations for these terms, leading to semantic errors. The proposed method utilizes non-parallel KGs to "hallucinate" high-quality training data, allowing the model to learn these mappings within its standard architecture.
Methodology
The proposed framework consists of three primary stages: entity induction, pseudo-sentence generation, and joint training.
1. Bilingual $K-D$ Entity Induction
The first step involves aligning the source and target KGs. Entities are represented as low-dimensional embeddings using the TransE algorithm. A set of "seed pairs"—entities appearing in both the KGs and a standard phrase translation table—is used to learn a linear transformation matrix ($W$). This matrix maps source KG embeddings into the target KG's semantic space by minimizing the distance between seed pairs. Once aligned, new translation pairs for $K-D$ entities are induced by identifying nearest neighbors across the two graphs.
2. Pseudo-Sentence Pair Generation
To integrate these induced pairs into the NMT model, the authors generate synthetic training data using "context transfer." If an induced pair (e.g., a rare chemical name) is semantically similar to a known seed pair (e.g., "aspirin"), the system identifies real training sentences containing "aspirin" and replaces it with the rare chemical name. This approach ensures the synthetic data maintains natural syntax, helping the model learn how to integrate new terms grammatically.
3. Joint Training
The NMT model (Transformer or RNN-based) is then trained on a mixture of the original parallel corpus and the newly generated pseudo-parallel data. To ensure the model effectively learns from the smaller volume of synthetic data, the pseudo-sentence pairs are over-sampled during the training process.
Experimental Results
The method was evaluated on Chinese-to-English (CN$\Rightarrow$EN) and English-to-Japanese (EN$\Rightarrow$JA) translation tasks across multiple domains, including Medical, Tourism, and General.
Quantitative Performance
The proposed approach consistently outperformed baseline Transformer and RNMT models. In the CN$\Rightarrow$EN Medical task, the method achieved a BLEU score of 15.69, a 1.12-point improvement over the baseline. The gains were even more pronounced for sentences containing $K-D$ entities, where BLEU scores rose from 13.46 to 15.15.
Hyper-parameter Sensitivity
The performance of the model is sensitive to the hyper-parameter $\lambda$, which controls the distance threshold for inducing pairs and generating sentences. As shown in the figure below, there is a clear "sweet spot" where the balance between the quantity of pseudo-data and the precision of the induced translations is optimized.
Qualitative Analysis
Qualitative results highlight the model's ability to handle rare terms. For instance, a baseline model translated a specific chemical term as "book of agriculture," whereas the KG-enhanced model correctly identified and translated it.
Conclusion
By leveraging the structured information within Knowledge Graphs, this research provides a robust method for handling rare and unseen entities in NMT. The use of semantic alignment and context-aware data augmentation allows models to generalize beyond their initial training data. Future work may explore non-linear mapping techniques for KG alignment and methods to reduce dependency on initial seed pairs in low-resource settings.
Summary updated