End-to-End Lexically Constrained Machine Translation for Morphologically Rich Languages

Josef Jon, João Paulo Aires, Dusan Varis, Ondřej BojarACL 2021

TL;DR

This paper tackles the challenge of ensuring specific words appear in machine translations, especially for languages with complex grammar (morphologically rich languages). They propose a novel approach by training neural machine translation models with lemmatized (base form) constraints, allowing the model to correctly inflect them based on context, significantly reducing agreement errors.

What Problem Does This Paper Address?

Machine translation (MT) systems, particularly Neural Machine Translation (NMT), are powerful but can struggle with specific user requirements. One such requirement is "lexical constraining," where a user wants to ensure certain words or phrases appear in the translated output. This is crucial for domain adaptation, translating named entities, or enforcing specific terminology. However, existing methods often fail to handle word inflections correctly, especially in morphologically rich languages like Czech. This means a constraint like "plan" might be forced into the translation as "plan," even if the context demands "planned" or "planning." This leads to grammatical errors and unnatural translations. The authors highlight that a significant portion of errors in constrained translation stems from these agreement issues.

Methods

The core of this paper's contribution lies in its approach to handling word inflections within lexically constrained NMT. Instead of providing the exact target word form as a constraint, they use the lemmatized form (the base, uninflected form of the word).

Here's a breakdown of their approach:

  • Learned Constraining: They integrate constraints directly into the training process by appending them to the source sentence, separated by special tokens (<sep>, <c>). This is a "learned" approach, meaning the model learns to utilize these constraints during training.
  • Lemmatized Constraints: The key innovation is using lemmatized constraints. This allows the NMT model, with its inherent language modeling capabilities, to infer the correct surface form (inflection) of the constraint based on the surrounding translated text.
  • Input Preparation: For training, they generate constraints by either randomly sampling subsequences from reference translations or by using dictionary lookups. For test time, they primarily rely on dictionary lookups.
  • Model Architecture: They use the Transformer architecture, trained with standard parameters, and employ techniques like alternating between authentic and back-translated data for robust training.

Datasets

  • Training Data: English-Czech parallel data from CzEng 2.0 (61M sentences) and back-translated Czech monolingual sentences (51M).
  • Validation/Test Data: Newstest-2019 for validation and Newstest-2020 for testing.
  • Domain-Specific Data: Europarl corpus for evaluating domain-specific terminology integration.
  • Rare Words Data: A subset of WMT 2007-2020 news test sets for evaluating rare word translation.
  • Out-of-Domain Data: A sublease agreement translated from Czech to English for a case study.
  • Constraint Sources: Wiktionary and a proprietary commercial dictionary were used to generate constraints.

Key Insights and Ideas

  • Lemmatization is Key for Inflection: The most significant insight is that using lemmatized constraints, rather than surface-form constraints, is crucial for achieving correct word inflections in morphologically rich languages. This leverages the NMT model's ability to generate contextually appropriate word forms.
  • Reference-Based Evaluation Can Be Misleading: The authors found that automatic evaluation metrics, which compare against a reference translation, can sometimes misclassify correct translations as errors. This happens when the model produces a valid, contextually correct translation that differs from the specific surface form in the reference.
  • Learned Constraining Outperforms Constrained Decoding for Inflection: While constrained decoding methods can enforce word presence, they struggle with inflection. The learned approach, especially with lemmatized constraints, proves more effective for handling grammatical agreement.
  • "Oracle" Setting Shows Potential: By using the reference translation to determine the "correct" surface form constraint at test time, they demonstrate the upper bound of constraint integration, showing high coverage when inflection is not an issue. This highlights the problem's severity when the model has to infer the correct form.

Key Results

  • Reduced Agreement Errors: The lemmatized constraint approach significantly reduced agreement errors, which constituted a large portion of errors in baseline constrained models.
  • Improved Coverage: The lemmatized models achieved higher surface form coverage when compared to models trained with non-lemmatized constraints, especially in the "diff" test set (where constraint forms differ from the reference).
  • Comparable Overall Quality: The proposed method improved constraint integration without negatively impacting overall translation quality (BLEU scores) or introducing new errors.
  • Effective for Rare Words: The approach also showed promise in translating rare words, even when dictionary translations had multiple variants.
  • Constraint Placement: The models were shown to place constraints at correct positions in the output, not just at the end.

Limitations and Future Work

  • Reference-Based Evaluation Artifacts: As mentioned, the automatic evaluation can misclassify correct translations as errors when the model deviates from the reference surface form. This is particularly true for lemmatized models that produce contextually correct, but different, inflections.
  • Dictionary Coverage: The performance of dictionary-based constraint generation is dependent on the dictionary's coverage. While a commercial dictionary offered better results, it impacts reproducibility.
  • Future Work: The authors plan to explore solving the problem of choosing the correct translation term from multiple variants in dictionary lookups. They also suggest further research into factored approaches for constraint integration, which showed promising results.

Summary updated

BibTeX
@inproceedings{jon2021end,
  author = {Josef Jon and João Paulo Aires and Dusan Varis and Ondřej Bojar},
  title = {{End-to-End Lexically Constrained Machine Translation for Morphologically Rich Languages}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure