TL;DR
This paper presents a hybrid machine translation system that combines Statistical Machine Translation (SMT) with a Neural Machine Translation (NMT) model for Lithuanian to English translation. By using NMT to post-edit SMT outputs, the system achieved a BLEU score of 17.6 on the WMT2019 News Translation Task, demonstrating the benefit of this combined approach.
What Problem Does This Paper Address?
Machine Translation (MT) aims to automatically translate text between languages, a task that is inherently complex due to the nuances of syntax and semantics. While both Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) have made significant strides, each has its limitations. SMT, while popular, can struggle with fluency and handling out-of-vocabulary words. NMT, on the other hand, has shown improvements in fluency and context handling but can be data-hungry and computationally intensive. This paper addresses the challenge of improving translation quality by exploring a hybrid approach that leverages the strengths of both SMT and NMT to overcome their individual weaknesses.
Methods
The authors propose a hybrid system that operates in a pipeline. First, a Statistical Machine Translation (SMT) model is trained using the Moses Toolkit. This SMT model is then used to translate the Lithuanian source sentences into English. Subsequently, a word embedding-based Neural Machine Translation (NMT) model is trained. This NMT model learns to map the English outputs from the SMT system to the gold-standard English translations. This post-editing step by the NMT model aims to refine the SMT output.
The NMT component specifically utilizes a sequence-to-sequence (seq2seq) architecture with a single LSTM layer for both the encoder and decoder. The encoder processes the Lithuanian input sequence, and the decoder generates the English output sequence, conditioned on the encoder's output and the previously generated target words.
Datasets
The primary dataset used in this work is the parallel corpus provided for the WMT2019 News Translation Shared Task. This corpus consists of 9,62,022 Lithuanian-English sentence pairs.
- Training: 7,62,022 sentence pairs were used to train the SMT system.
- NMT Training/SMT Testing: 2,00,000 sentence pairs were used for testing the SMT system and subsequently for training the NMT system.
- Testing: 10,000 Lithuanian sentences were used for testing the final hybrid model.
The paper also mentions the use of an English monolingual corpus for building the language model for the SMT system.
Key Insights and Ideas
- Hybridization is beneficial: The core idea is that combining SMT and NMT can yield better results than using either system in isolation. The NMT model acts as a sophisticated post-editor for the SMT output.
- NMT as a Refiner: The NMT model is trained to learn the mapping between the SMT's English output and the correct English translation. This suggests that NMT can effectively correct errors or improve the fluency of SMT outputs.
- Leveraging existing tools: The use of established tools like Moses Toolkit for SMT and KenLM for language modeling demonstrates a practical approach to building a competitive system.
- Word embeddings for NMT: Employing word embeddings in the NMT model is a standard but crucial component for capturing semantic relationships between words.
Key Results
The hybrid system achieved a BLEU score of 17.6 on the WMT2019 News Translation Task for Lithuanian to English. Other evaluation metrics reported include:
- BLEU-cased: 16.6
- TER: 0.762
- BEER 2.0: 0.497
- CharactTER: 0.718
The paper notes that testing with 10k Lithuanian sentences resulted in an accuracy of 21.6, though this specific number might refer to an intermediate or different evaluation setup than the final reported BLEU score.
Limitations and Future Work
Limitations: * The paper mentions that the NMT model uses a single LSTM layer for both encoder and decoder. This might limit its capacity to capture complex linguistic phenomena compared to deeper architectures. * The reported BLEU score of 17.6, while an improvement over individual SMT or NMT systems (implied), is still relatively low, indicating significant room for improvement in Lithuanian-to-English translation quality.
Future Work: * The authors plan to explore using more LSTM layers in their NMT model to enhance its representational power. * They also intend to investigate a pipeline where both SMT and NMT systems are trained incrementally, suggesting a more integrated approach to hybrid training.
Summary updated