Cognates Can Improve Statistical Translation Models

Grzegorz Kondrak, Daniel Marcu, Kevin KnightNAACL 2003

TL;DR

This paper explores how to leverage "cognates" – words with similar spelling or pronunciation across languages – to boost the quality of statistical machine translation (SMT). By treating these cognates as additional training data, the researchers demonstrate improvements in word alignment and, consequently, translation accuracy, without requiring external dictionaries.

What Problem Does This Paper Address?

Statistical Machine Translation (SMT) models, like those based on IBM's Model 4, learn translation patterns from large parallel corpora (bitexts). However, these models traditionally treat words in isolation, focusing on their statistical co-occurrence. This can be a limitation, especially when dealing with languages that share many similar-looking words due to genetic relationships or borrowing. Identifying and utilizing these "cognates" – words like "night" (English) and "nacht" (German), or "sprint" (English) and "supurinto" (Japanese) – presents an opportunity to enrich the training data and improve translation quality, particularly when bilingual dictionaries are scarce.

Methods

The core idea is to augment the training bitext with identified cognate pairs. The authors experimented with three similarity measures to detect these cognates:

  • Simard's condition: Two words are considered cognates if they are at least four characters long and their first four characters match.
  • Dice's coefficient: Measures the overlap of character bigrams between two words.
  • Longest Common Subsequence Ratio (LCSR): Calculates the ratio of the longest common subsequence of characters to the length of the longer word.

Once potential cognates are identified using these measures on a tokenized and sentence-aligned bitext, these pairs are appended to the training corpus. The authors explored different "duplication factors" (how many times the cognate list is added) and thresholds for the similarity measures to find the optimal way to incorporate this information. The standard IBM Model 4 SMT system, trained using the GIZA toolkit, was then used to build translation models with and without this cognate augmentation.

Datasets

  • Hansards: A parallel corpus used for experiments on word alignment quality. A subset of 500 manually aligned sentences was used for direct evaluation, and a larger portion (50,000 sentences) for training SMT models.
  • Europarl: A tokenized and sentence-aligned multilingual corpus extracted from the Proceedings of the European Parliament. This corpus, covering eleven official European Union languages, was used for evaluating translation quality across multiple language pairs. Training data consisted of subsets of approximately 20,000 aligned sentences per language pair, while test data comprised 1755 unseen sentences. An additional English language model was trained on a larger set of 700,000 sentences.

Key Insights and Ideas

  • Cognates as implicit dictionaries: Cognates act as a form of "free" bilingual dictionary, providing valuable translation links without explicit human curation or reliance on external resources.
  • Robustness of SMT algorithms: Statistical machine translation algorithms appear robust enough to handle a certain number of "dubious" cognates (identified with lower similarity thresholds) without significantly degrading translation quality. In fact, adding more, even less certain, cognates sometimes yielded better results than relying solely on high-confidence pairs.
  • Word alignment improvement: The primary mechanism through which cognates improve translation is by enhancing word alignment quality. This leads to more accurate mapping of words between languages.
  • Rare word alignment: Cognate information helps reduce the tendency of rare words to align to multiple unrelated words, leading to more precise alignments.

Key Results

  • Word Alignment: A 10% reduction in word alignment error rate was observed when cognate information was incorporated, improving both precision and recall. The alignment of identical words increased from 93.6% to 97.2%.
  • Translation Quality (BLEU Scores): Experiments on the Europarl corpus showed statistically significant improvements in BLEU scores when cognates were added, particularly when using duplication factors greater than one. For example, with a duplication factor of 2, BLEU scores increased across various similarity measures.
  • Manual Evaluation: A manual evaluation of translations confirmed that the cognate-augmented models produced a higher percentage of completely acceptable translations compared to the baseline.

Limitations and Future Work

  • Crude approach: The authors acknowledge that appending cognate pairs directly to the training data is a relatively "crude" approach.
  • Future work: They plan to develop methods for incorporating cognate information directly into the SMT training algorithm itself, rather than just augmenting the data. They also suggest exploring more sophisticated word similarity measures for even better results.

Summary updated

BibTeX
@inproceedings{kondrak2003cognates,
  author = {Grzegorz Kondrak and Daniel Marcu and Kevin Knight},
  title = {{Cognates Can Improve Statistical Translation Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2003},
}

Loading related content...

Figure