LIT Team’s System Description for Japanese-Chinese Machine Translation Task in IWSLT 2020

Yimeng Zhuang, Yuan Zhang, Lijie Wang2020 · DOI

Abstract

This paper describes the LIT Team's submission to the IWSLT2020 open domain translation task, focusing primarily on Japanese-to-Chinese translation direction. Our system is based on the organizers' baseline system, but we do more works on improving the Transformer baseline system by elaborate data preprocessing. We manage to obtain significant improvements, and this paper aims to share some data processing experiences in this translation task. Large-scale back-translation on monolingual corpus is also investigated. In addition, we also try shared and exclusive word embeddings, compare different granularity of tokens like sub-word level. Our Japanese-to-Chinese translation system achieves a performance of BLEU=34.0 and ranks 2nd among all participating systems.

Introduction

Machine translation between Japanese (JA) and Chinese (ZH) presents unique linguistic challenges. Despite sharing a common logographic heritage (Kanji and Hanzi), the two languages differ significantly in grammar, syntax, and word order. In the context of the IWSLT 2020 open-domain task, the primary hurdle was the high level of noise in web-crawled datasets, which often contain mismatched pairs, duplicates, or incorrect language tags.

The LIT Team (Samsung Research China) addressed these challenges by prioritizing data quality over raw volume. By implementing aggressive multi-stage filtering and leveraging large-scale back-translation, the team transformed a noisy baseline into a high-performing system that secured 2nd place in the competition.

Methodology

The system architecture is based on the Transformer model, enhanced with specific modifications for the JA-ZH task.

Model Architecture

  • Bigger Transformer: To increase model capacity, the inner dimension of the feed-forward network was expanded from 4096 to 8192. To mitigate overfitting, ReLU dropout was increased from 0.1 to 0.3.
  • Relative Position Representation: The team replaced absolute position embeddings with Relative Position Attention. This allows the model to more effectively capture sequential relationships, which is crucial given the structural differences between Japanese and Chinese.

Data Preprocessing and Filtering

The core of the team's success lay in "elaborate data filtering." They applied a series of strict rules to the provided corpora:

  1. Language Identification: Using fastText to ensure sentence pairs were actually in the target languages.
  2. Length Constraints: Removing pairs with extreme length ratios or those exceeding 256 tokens.
  3. Character Matching: Removing pairs that lacked common characters, a specific heuristic for JA-ZH pairs.
  4. Rule-based Cleaning: Removing HTML tags, duplicate sentences, and sentences with excessive punctuation.

Data Augmentation

To leverage 200 million Chinese monolingual sentences, the team employed Back-Translation (BT). A ZH→JA model was trained to generate synthetic Japanese source sentences. To improve robustness, noise (such as word deletion or swapping) was introduced into the synthetic data before training the final JA→ZH model.

Datasets and Data Reduction

The training data consisted of four parts (A, B, C, and D) provided by the organizers. The impact of the filtering process was most evident in Part C, a massive but noisy web corpus.

Corpus statistics showing original vs. re-filtered data sizes.

As shown in the data statistics, the filtering process was highly selective. Part C was reduced from 161.5 million pairs to just 10 million high-quality pairs. This "quality over quantity" approach proved that removing noisy data is more beneficial than training on a larger, lower-quality set.

Results and Discussion

The iterative application of these techniques led to consistent improvements in translation quality, measured by BLEU scores.

Incremental performance improvements across different system configurations.

The baseline system, trained only on clean data, achieved a Test BLEU of 22.0. Adding filtered data and then applying stricter "Re-Filtering" rules significantly boosted the Dev BLEU. The final system, incorporating the "Bigger Transformer" architecture, Relative Position (RP) attention, and Back-Translation (BT), reached a final Test BLEU of 34.0.

Insights

  • Filtering vs. Alignment Tools: The team found that standard tools like fast_align could be counterproductive, as they often penalized long, complex sentences that were actually high-quality.
  • Architectural Synergy: The combination of increased model capacity and relative position embeddings allowed the model to better utilize the cleaned and augmented datasets.

Conclusion

The LIT Team's approach demonstrates that meticulous data engineering is as critical as architectural innovation in modern NMT. By aggressively filtering noisy web data and strategically using back-translation, they achieved a top-tier result. Future work will focus on model ensembling and more sophisticated post-processing techniques to further refine translation outputs.

Summary updated

Abstract

This paper describes the LIT Team's submission to the IWSLT2020 open domain translation task, focusing primarily on Japanese-to-Chinese translation direction. Our system is based on the organizers' baseline system, but we do more works on improving the Transformer baseline system by elaborate data preprocessing. We manage to obtain significant improvements, and this paper aims to share some data processing experiences in this translation task. Large-scale back-translation on monolingual corpus is also investigated. In addition, we also try shared and exclusive word embeddings, compare different granularity of tokens like sub-word level. Our Japanese-to-Chinese translation system achieves a performance of BLEU=34.0 and ranks 2nd among all participating systems.
BibTeX
@misc{zhuang2020lit,
  author = {Yimeng Zhuang and Yuan Zhang and Lijie Wang},
  title = {{LIT Team’s System Description for Japanese-Chinese Machine Translation Task in IWSLT 2020}},
  year = {2020},
  doi = {10.18653/v1/2020.iwslt-1.12},
}
Figure