Translating Unknown Words by Analogical Learning

Philippe Langlais, Alexandre PatryEMNLP 2007 · OpenReview

Abstract

Philippe Langlais, Alexandre Patry. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL).

  1. This paper introduces a method for translating unknown (out-of-vocabulary) words in machine translation by leveraging analogical learning. By solving proportional analogies—the linguistic equivalent of "A is to B as C is to D"—the system can infer the translation of a new word based on patterns found in existing bilingual lexicons without requiring explicit morphological rules.

Introduction

Machine Translation (MT) systems frequently encounter "out-of-vocabulary" (OOV) words—terms not present in their training data—which significantly degrade translation quality. While proper names can often be copied directly, "ordinary" words like inflected verbs or plural nouns remain a challenge. This paper addresses the translation of these ordinary words by exploiting structural relationships between words. Instead of relying on complex, language-specific morphological analyzers, the authors propose using patterns already present in a bilingual lexicon to "guess" the translation of a new word.

Methodology

The core of the approach is the proportional analogy, denoted as $[A : B = C : D]$ (e.g., walk : walking = surf : surfing). The authors propose a three-step inference procedure for an unknown source word $S$:

  1. Source Discovery: Identify triplets of known words $(A, B, C)$ from the training data that form a formal analogy with the unknown word $S$ (i.e., $[A : B = C : S]$).
  2. Target Projection: Look up the known translations $(A', B', C')$ for each triplet in the bilingual lexicon.
  3. Target Resolution: Solve the analogy in the target language $[A' : B' = C' : ?]$ to find the candidate translation $T$.
The three-step inference procedure for translating unknown words.

To solve these equations at the character level, the authors implemented a solver based on edit distance and alignment synchronization. It identifies character sequences inserted or deleted between $A$ and $B$ and applies the same transformation to $C$ to produce $D$. To manage the $O(N^3)$ computational complexity of searching for triplets, a heuristic limits the search to words $A$ and $B$ within a small edit-distance neighborhood of $S$.

Experimental Setup

The method was evaluated on translation tasks from French, Spanish, and German into English using the Europarl corpus. Two evaluation sets were used: test-in (2,000 sentences from the Europarl domain) and test-out (1,064 sentences from Project Syndicate news editorials) to assess out-of-domain performance. A target vocabulary of ~466,000 English words was used to filter out invalid candidates generated by the solver.

Results and Discussion

The analogical learning approach demonstrated high precision, proposing valid translations for 80% of ordinary unknown words in the French-to-English task. It significantly outperformed baselines that relied on simple similarity or projection.

Examples of translations generated for French out-of-vocabulary words.

Key Insights

  • Implicit Morphology: The system effectively "learns" morphology on the fly. By observing relationships like activités : activité = futilités : futilité, it can correctly translate futilités as trivialities.
  • Domain Independence: The method performed well on out-of-domain text (test-out), suggesting that morphological patterns are more stable across domains than specific vocabulary.
  • Phrase Translation: The approach extends to short phrases (up to 5 words), capturing 1-to-many relationships, such as translating a single French future-tense verb into a multi-word English phrase.

Limitations

Despite its success, the method faces challenges with computational cost due to the quadratic search space. It is also less effective for proper names, compound words (common in German), or numerical expressions. Furthermore, as the training set grows, the risk of "spurious analogies" (accidental patterns) increases, potentially leading to noisy translations.

Conclusion

Analogical learning provides a robust, language-independent framework for translating OOV words by leveraging the relational structure of existing lexicons. By bypassing the need for explicit morphological rules, it offers a scalable way to improve MT performance, particularly in out-of-domain scenarios where OOV words are prevalent.

Summary updated

BibTeX
@inproceedings{langlais2007translating,
  author = {Philippe Langlais and Alexandre Patry},
  title = {{Translating Unknown Words by Analogical Learning}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2007},
  url = {https://openreview.net/forum?id=sFvdyhsIXis},
}
Figure