Abstract
This paper describes the Global Tone Communication Co., Ltd.'s submission for the WMT18 shared news translation task in the English-to-Chinese direction. Our system achieved the top BLEU score of 43.8 by prioritizing rigorous data filtering over architectural complexity. We applied a multi-stage cleaning process involving rule-based checks, language models, and translation models to both provided parallel data and back-translated synthetic data. Additionally, we validated the effectiveness of the Annealing Adam optimizer and greedy ensemble decoding. Our results demonstrate that data quality is the primary driver of performance in modern Transformer-based translation systems.
Introduction
The WMT shared tasks represent the state-of-the-art in machine translation. For the 2018 English-to-Chinese news task, researchers faced two primary challenges: managing highly noisy large-scale parallel corpora and adapting established optimization techniques to the then-emerging Transformer architecture. This study explores how aggressive data preprocessing and specific training refinements can push the boundaries of translation accuracy.
Methodology
The core of our system is the Transformer architecture, implemented using the Marian toolkit. While the architecture is standard, our approach distinguishes itself through a rigorous three-tiered data filtering pipeline designed to eliminate misalignments and low-quality translations.
Data Filtering Strategy
Our filtering pipeline targeted both original parallel data and synthetic back-translated data. We employed three main methods:
- Rule-based Filtering: Discarding sentence pairs with significant discrepancies in punctuation or length.
- Language Model (LM) Filtering: Using cross-entropy scores to identify and remove statistically "unnatural" sentences.
- Translation Model (TM) Filtering: Scoring the probability of a target sentence given the source to remove low-probability pairs.
Beyond filtering, we utilized Back-translation to augment our training set and employed Annealing Adam, a variant of the Adam optimizer that dynamically adjusts the learning rate to improve convergence during training.
Results and Analysis
Our experiments confirmed that the Transformer architecture significantly outperforms previous RNN-based models. However, the most substantial gains came from data refinement and optimization techniques.
The Impact of Data Quality
As shown in our data statistics, we removed nearly 5 million parallel sentences and roughly half of the synthetic back-translated data. Despite this reduction in volume, the "cleaner" dataset led to a marked improvement in translation quality. This supports the "quality over quantity" hypothesis in neural machine translation.
Optimization and Ensembling
The transition to Transformers rendered some traditional RNN techniques obsolete. For instance, Right-to-Left (R2L) reranking, which typically boosts RNN performance, actually decreased our BLEU score. Conversely, Annealing Adam and Greedy Ensemble decoding—where models are added to an ensemble only if they improve performance—remained highly effective.
Conclusion
The success of our WMT18 submission highlights that while the Transformer provides a powerful baseline, the "ceiling" of performance is largely determined by data cleanliness. Our final system achieved a BLEU score of 43.8, the highest in the competition. Future work will focus on making these computationally expensive filtering processes more efficient and investigating why certain ensemble techniques fail to translate from RNNs to attention-based architectures.
Summary updated