Adaptation Using Out-of-Domain Corpus within EBMT

Takao Doi, Eiichiro Sumita, Hirofumi YamamotoNAACL 2003

TL;DR

This paper tackles the challenge of adapting Example-Based Machine Translation (EBMT) systems to new domains when only a small in-domain bilingual corpus is available. They demonstrate that incorporating a larger, out-of-domain bilingual corpus and a language model of the target language can significantly improve translation quality, even outperforming using a larger in-domain corpus alone.

What Problem Does This Paper Address?

Example-Based Machine Translation (EBMT) is known for its adaptability to new domains. The typical approach involves preparing a bilingual corpus for the target domain. However, when this in-domain corpus is small, the translation quality suffers. This paper investigates methods to boost EBMT performance in such low-resource scenarios by leveraging readily available out-of-domain data and monolingual target language data.

Methods

The authors explore three adaptation methods for EBMT:

  1. Merging Equally: Combining the in-domain and out-of-domain corpora and using them without distinction.
  2. Merging with Preference for In-domain Corpus: Merging both corpora but prioritizing in-domain examples when multiple examples have the same similarity score.
  3. Using Language Model (LM): This method involves building a language model of the target language from a monolingual corpus. During example retrieval, out-of-domain examples are scored based on the probability of their target sentences according to this LM. The retrieval process then considers both in-domain and out-of-domain examples, with a preference for in-domain examples when similarities are equal.

Datasets

  • Out-of-domain corpus: Basic Travel Expression Corpus (BTEC), described as the "BE-corpus." This corpus is significantly larger than the in-domain corpus.
  • In-domain corpus: A telephone conversation corpus (TEL). This corpus was split into a test set (1,653 sentence pairs) and a training set (9,918 sentence pairs).
  • Language Model: A word trigram model of the English sentences from the TEL training set was used.

Key Insights and Ideas

  • Out-of-domain data is valuable: Even a large out-of-domain corpus can be beneficial, and when merged equally with a small in-domain corpus, it leads to better results than using only the small in-domain corpus.
  • Target language LM is crucial: Incorporating a language model of the target language significantly improves translation quality, especially when the in-domain corpus is very small.
  • LM from larger corpora is better: Using an LM trained on the entire target language training set yields slightly better results than using an LM trained on only a partial set.
  • Adaptation is more effective with smaller in-domain corpora: The benefits of the proposed adaptation methods are more pronounced when the in-domain bilingual corpus is limited.

Key Results

The experiments, evaluated using BLEU and NIST scores, showed the following:

  • BLEU Scores:
    • Using 4,000 in-domain examples achieved better scores than using 152,172 out-of-domain examples.
  • Merging equally with out-of-domain data outperformed using only out-of-domain data.
  • Merging with in-domain preference was better than equal merging.
  • Using the LM method consistently yielded the best results across different sizes of in-domain data.
  • When no in-domain examples were used, the LM method showed a relatively large effect.
  • NIST Scores: Similar tendencies were observed as with BLEU scores, with the LM method generally performing best.

Limitations and Future Work

The authors acknowledge that the gains from using the LM were not always significant. They suggest future work should involve:

  • Continuing experiments with other corpora and under various conditions.
  • Investigating the use of low-quality in-domain corpora, as their current work implicitly assumes a high-quality in-domain corpus.

Summary updated

BibTeX
@inproceedings{doi2003adaptation,
  author = {Takao Doi and Eiichiro Sumita and Hirofumi Yamamoto},
  title = {{Adaptation Using Out-of-Domain Corpus within EBMT}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2003},
}
Figure