Abstract
A translation memory (TM) is proved to be helpful to improve neural machine translation (NMT). Existing approaches either pursue the decoding efficiency by merely accessing local information in a TM or encode the global information in a TM yet sacrificing efficiency due to redundancy. We propose an efficient approach to making use of the global information in a TM. The key idea is to pack a redundant TM into a compact graph and perform additional attention mechanisms over the packed graph for integrating the TM representation into the decoding network. We implement the model by extending the state-of-the-art NMT, Transformer. Extensive experiments on three language pairs show that the proposed approach is efficient in terms of running time and space occupation, and particularly it outperforms multiple strong baselines in terms of BLEU scores.
Introduction
In professional translation workflows, Translation Memory (TM) databases are indispensable tools that allow translators to leverage previously translated sentence pairs. Integrating these databases into Neural Machine Translation (NMT) systems has been a significant research goal. However, current methods face a critical trade-off: local-pattern methods (like n-gram matching) are efficient but lack global structural context, while global-encoding methods capture sentence structure but suffer from extreme computational redundancy. Because retrieved TM sentences often share significant vocabulary, standard encoding methods repeatedly process the same words, leading to slow decoding and high memory consumption. This paper introduces G-TFM (Graph-based Transformer), a model designed to capture global TM context with high efficiency.
Methodology: Graph-Based TM Integration
The core innovation of G-TFM is the transformation of multiple retrieved TM sentences into a single, compact directed graph (a confusion network). This approach addresses redundancy at the architectural level.
- Target-Side Focus: The model prioritizes the TM's target sentences. Since the input source sentence is already processed by the main encoder, encoding the TM's source side is largely redundant and is omitted to save memory.
- Graph Packing: Instead of sequential encoding, the top retrieved target sentences are merged into a graph. For example, if a specific term appears across multiple TM entries, it is represented as a single node, with sentences represented as paths through the graph.
- Graph Attention Network: A Graph Attention mechanism is employed to encode this compact structure, allowing the model to learn which parts of the TM are most relevant to the current translation step.
- Modified Decoder: The standard Transformer decoder is augmented with a dedicated Multi-Head Attention sub-layer and residual connection to attend to the encoded graph, enabling the dynamic integration of syntactic and semantic clues.
Experimental Setup
The model was evaluated using the JRC-Acquis corpus, which contains parallel legislative texts from European Union Law. Experiments were conducted across six bidirectional tasks involving English-French (en-fr), English-Spanish (en-es), and English-German (en-de). The setup utilized Byte-pair Encoding (BPE) with a 20K vocabulary and a development/test split of 3,000 samples each.
Results and Discussion
G-TFM demonstrates a superior balance of translation quality and computational efficiency compared to both standard Transformers and previous TM-augmented models.
- Accuracy: On the Spanish-to-English (es-en) task, G-TFM achieved a BLEU score of 66.21, significantly higher than the baseline Transformer (62.68) and the strong P-TFM baseline (64.30).
- Efficiency: As shown in the performance comparison, G-TFM achieves a ~7.5x speedup over global-encoding methods like SEG-TFM. By merging redundant words, the graph approach reduces the average number of encoded words from 374 to 129 per TM retrieval.
- Contextual Reordering: A key qualitative finding was the model's ability to handle long-distance reordering. By retaining positional information within the graph, G-TFM correctly placed subordinate clauses that baseline models typically misplace.
Conclusion
G-TFM provides a robust solution to the redundancy problem in TM-augmented NMT. By utilizing graph packing and graph attention, the model achieves state-of-the-art accuracy while maintaining the efficiency required for real-world applications. Future work may explore end-to-end training of the retrieval mechanism to further optimize the integration of external translation memories.
Summary updated