Towards better Chinese-centric neural machine translation for low-resource languages

Bin Li, Yixuan Weng, Fei Xia, Hanjun Deng

Abstract

Neural Machine Translation (NMT) typically relies on high-resource language pairs, often using English as a central hub. This paper details a winning approach for the 2021 iFLYTEK AI Developer Competition, focusing on Chinese-centric translation for low-resource languages like Malaysian and Indonesian. The authors introduce a robust pipeline incorporating "In-trust" loss to mitigate data noise and a multi-stage bilingual curriculum learning strategy. Their system achieved first place, significantly outperforming established baselines like mBART and Google Translate, reaching a BLEU-avg of 25.83.

Introduction

Most NMT research focuses on high-resource languages where massive parallel datasets are available. When low-resource languages are addressed, they are usually paired with English. This creates a "Chinese-centric" gap, particularly for regional languages like Malaysian and Indonesian. Translating these pairs is challenging due to data scarcity, inherent noise in web-crawled datasets (e.g., encoding errors, poor translations), and the difficulty models face with complex grammatical structures in low-resource settings.

Methodology

The researchers employed a "divide and conquer" strategy to build their champion NMT system. The overall pipeline consists of four main stages: corpus acquisition, data preprocessing, backbone model fine-tuning, and post-processing.

Overview of the NMT pipeline including data preprocessing, backbone model (mBART), and post-processing.

Backbone and Data Enhancement

The system uses mBART, a large-scale pre-trained multilingual model, as its foundation. To overcome data scarcity, the authors used back-translation via Google’s API and word embedding replacement using monolingual data to generate synthetic training examples.

Incomplete-Trust (In-trust) Loss

A primary technical contribution is the In-trust loss function. Unlike standard cross-entropy loss, which can cause models to overfit to noisy data, In-trust loss is "noise-aware." It allows the model to learn from the data while maintaining a level of uncertainty, effectively preventing the learning of errors present in the training set.

Bilingual Curriculum Learning and Re-ranking

Training proceeded in stages: first on similar language families (Indonesian and Malaysian), then on short sentences, and finally on long, complex sentences. For post-processing, a secondary MBERT model was used for contrastive re-ranking, selecting the most accurate translation from multiple candidates.

Results and Discussion

The proposed system won first place in the iFLYTEK competition, achieving a BLEU-avg of 25.83. This outperformed the mBART baseline (24.52) and even surpassed the Google Translation Engine in specific directions, such as Malaysian-to-Chinese (23.53 vs. 21.87 BLEU).

Ablation study showing the incremental BLEU-avg improvements provided by each component of the proposed method.

As shown in the ablation study, while every technique contributed to the final score, the In-trust loss provided the single largest performance boost (+0.56 BLEU). This confirms that handling data noise mathematically is as critical as the volume of data itself. Furthermore, leveraging language family similarities during curriculum learning provided a stronger foundation for the model.

Conclusion

This work demonstrates that high-quality Chinese-centric NMT for low-resource languages is achievable through sophisticated noise handling and structured learning strategies. While the current system involves computational overhead due to multi-stage training and re-ranking, it sets a new benchmark for regional language translation. Future work will focus on scaling these methods to other language pairs and exploring counterfactual data to further enhance model robustness.

Summary updated

Abstract

Highlights•Focusing on Chinese low-resource neural translation for further development.•Presenting new ideas for translation by enhancing pre-trained language models.•Self-developing optimization loss to further improve translation quality.•The proposed system outperforms the existing translation engine.
BibTeX
@article{li2024towards,
  author = {Bin Li and Yixuan Weng and Fei Xia and Hanjun Deng},
  title = {{Towards better Chinese-centric neural machine translation for low-resource languages}},
  journal = {Comput. Speech Lang.},
  year = {2024},
  doi = {10.1016/j.csl.2023.101566},
}
Figure