Improving Mongolian-Chinese Neural Machine Translation with Morphological Noise

Yatu Ji, Hongxu Hou, Junjie Chen, Nier WuACL 2019 · DOI

Abstract

Neural Machine Translation (NMT) often struggles with agglutinative languages like Mongolian due to the "unknown word" (UNK) problem caused by high morphological complexity. This paper proposes a novel framework, VS-GAN (Value Screener Generative Adversarial Network), which treats complex word suffixes as "morphological noise." By utilizing a Value Iteration Network to filter training samples, the model focuses on difficult examples, leading to improved translation quality. Our results demonstrate significant gains in BLEU scores and a substantial reduction in both training time and the frequency of UNK tokens in Mongolian-to-Chinese translation tasks.

Introduction

In NMT, models typically rely on a fixed vocabulary. For agglutinative languages, where words are formed by appending multiple suffixes to a root to indicate grammatical features (tense, case, mood), the number of possible word forms is vast. When a model encounters a rare root-suffix combination, it often outputs an "unknown" (<unk>) token. This issue persists even with subword segmentation techniques like Byte Pair Encoding (BPE), as they may fail to capture the underlying semantic relationships between different morphological forms.

The authors argue that these morphological variations act as a form of "noise" that confuses the decoder. To address this, they propose reframing the problem as a noise-handling task within a Generative Adversarial Network (GAN) framework.

Methodology: The VS-GAN Framework

The core of the proposed solution is the VS-GAN architecture. Unlike standard GANs, it introduces a third component to optimize the adversarial training process:

  1. Generator (G): An NMT model (tested with both LSTM and Transformer architectures) that generates translation hypotheses.
  2. Discriminator (D): A CNN-based classifier that distinguishes between human-translated "real" sentences and machine-generated "fake" ones.
  3. Value Screener: This novel module sits between G and D, employing a Value Iteration Network (VIN). It assesses the quality of generated translation batches. If a batch is of high quality ("too good"), the screener may skip the discriminator phase. This forces the discriminator to focus on "noisy" or challenging examples where the generator is currently underperforming.
The VS-GAN architecture featuring the Value Screener module between the Generator (G) and Discriminator (D).

To further improve robustness, the researchers implemented a Noise Addition Strategy. They manually segmented Mongolian words and added suffixes and case markers to the training data, creating pseudo-data that explicitly exposes the model to morphological variations during the adversarial training process.

Experimental Results

The models were evaluated on Mongolian-Chinese (M-C) datasets from the CLDC and CWMT2017 campaigns, totaling approximately 230,000 sentence pairs. The VS-GAN framework demonstrated superior performance across multiple metrics compared to baselines like MIXER and standard Transformers.

Translation Quality and Efficiency

As shown in the performance table below, VS-GAN consistently achieved higher BLEU scores across various noise configurations. Notably, on the most complex "Original & Suffixes & Case" dataset, the Transformer-based VS-GAN reached a BLEU score of 33.2, compared to the baseline Transformer's 30.5.

Comparison of BLEU scores and training times across different models and noise configurations.

Beyond quality, the Value Screener significantly improved training efficiency. For the Transformer experiments, the adversarial training phase for VS-GAN took only 18 hours, representing a 50% reduction compared to the 36 hours required by the standard BR-CSGAN.

UNK Reduction

A key success of the VS-GAN approach was the drastic reduction in the frequency of <unk> tokens. By treating morphology as noise and using the Value Screener to focus on these difficult cases, the model learned to decode complex word forms more accurately, effectively bridging the semantic gap that standard BPE often leaves behind.

Conclusion

This research highlights the effectiveness of treating linguistic complexity as a signal-to-noise problem. By introducing the Value Screener into a GAN framework, the authors provided a way to perform more targeted adversarial training, specifically addressing the morphological challenges of agglutinative languages. Future work will explore the integration of syntax trees and the dynamic modification of grammar trees within the GAN framework to further enhance translation accuracy for structurally complex languages.

Summary updated

BibTeX
@inproceedings{ji2019improving,
  author = {Yatu Ji and Hongxu Hou and Junjie Chen and Nier Wu},
  title = {{Improving Mongolian-Chinese Neural Machine Translation with Morphological Noise}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-2016},
}
Figure