Abstract
Neural Machine Translation (NMT) typically relies on high-resource language pairs, often using English as a central hub. This paper details a winning approach for the 2021 iFLYTEK AI Developer Competition, focusing on Chinese-centric translation for low-resource languages like Malaysian and Indonesian. The authors introduce a robust pipeline incorporating "In-trust" loss to mitigate data noise and a multi-stage bilingual curriculum learning strategy. Their system achieved first place, significantly outperforming established baselines like mBART and Google Translate, reaching a BLEU-avg of 25.83.
Introduction
Most NMT research focuses on high-resource languages where massive parallel datasets are available. When low-resource languages are addressed, they are usually paired with English. This creates a "Chinese-centric" gap, particularly for regional languages like Malaysian and Indonesian. Translating these pairs is challenging due to data scarcity, inherent noise in web-crawled datasets (e.g., encoding errors, poor translations), and the difficulty models face with complex grammatical structures in low-resource settings.
Methodology
The researchers employed a "divide and conquer" strategy to build their champion NMT system. The overall pipeline consists of four main stages: corpus acquisition, data preprocessing, backbone model fine-tuning, and post-processing.
Backbone and Data Enhancement
The system uses mBART, a large-scale pre-trained multilingual model, as its foundation. To overcome data scarcity, the authors used back-translation via Google’s API and word embedding replacement using monolingual data to generate synthetic training examples.
Incomplete-Trust (In-trust) Loss
A primary technical contribution is the In-trust loss function. Unlike standard cross-entropy loss, which can cause models to overfit to noisy data, In-trust loss is "noise-aware." It allows the model to learn from the data while maintaining a level of uncertainty, effectively preventing the learning of errors present in the training set.
Bilingual Curriculum Learning and Re-ranking
Training proceeded in stages: first on similar language families (Indonesian and Malaysian), then on short sentences, and finally on long, complex sentences. For post-processing, a secondary MBERT model was used for contrastive re-ranking, selecting the most accurate translation from multiple candidates.
Results and Discussion
The proposed system won first place in the iFLYTEK competition, achieving a BLEU-avg of 25.83. This outperformed the mBART baseline (24.52) and even surpassed the Google Translation Engine in specific directions, such as Malaysian-to-Chinese (23.53 vs. 21.87 BLEU).
As shown in the ablation study, while every technique contributed to the final score, the In-trust loss provided the single largest performance boost (+0.56 BLEU). This confirms that handling data noise mathematically is as critical as the volume of data itself. Furthermore, leveraging language family similarities during curriculum learning provided a stronger foundation for the model.
Conclusion
This work demonstrates that high-quality Chinese-centric NMT for low-resource languages is achievable through sophisticated noise handling and structured learning strategies. While the current system involves computational overhead due to multi-stage training and re-ranking, it sets a new benchmark for regional language translation. Future work will focus on scaling these methods to other language pairs and exploring counterfactual data to further enhance model robustness.
Summary updated