Resolving Out of Vocabulary Words in Machine Translation

This direction is split into other directions: lexical coverage gap (out-of-vocabulary bottleneck). Its last version stays readable and citable here.

Version 1 of 157 papers57 conceptsfrozen 2026-10-04

The "Closed World Assumption" is a fundamental limitation of early machine translation. Most systems are trained on a fixed parallel corpus—a set of sentences in Language A and their direct translations in Language B. If a user provides a word that never appeared in that training set, the system hits a wall. These are Out-of-Vocabulary (OOV) words.

In Statistical Machine Translation (SMT), an OOV word typically results in a "copy-through" (where the foreign word is left untranslated) or an omission. In Neural Machine Translation (NMT), it often results in a generic <UNK> token, which destroys the sentence's meaning. Researchers have developed several distinct strategies to bridge this gap, moving from simple dictionary lookups to sophisticated semantic synthesis.

Pivoting and Paraphrasing

If we don't have a direct translation for a phrase like "abandoned house" from Spanish to English, we might find that we do have a translation for "abandoned house" in a Spanish-French corpus, and a translation for that French phrase in a French-English corpus.

Philipp Koehn, Chris Callison-Burch, and Miles Osborne formalized this as pivoting in Improved Statistical Machine Translation Using Paraphrases. By using a third language as a bridge, they could "hallucinate" new phrase pairs that were never explicitly seen in the primary training data.

However, simple substitution can be dangerous. A synonym might have a different grammatical function. Chris Callison-Burch later showed in Syntactic Constraints on Paraphrases Extracted from Parallel Corpora that enforcing syntactic consistency—ensuring the paraphrase plays the same role in a sentence tree—dramatically improves translation quality. This idea was further scaled by Ramtin Mehdizadeh Seraj and Anoop Sarkar, who used graph propagation to spread known translations across a massive Multilingual Paraphrase Database.

Morphological Synthesis and Subwords

Many languages are "morphologically rich," meaning they create new words by adding prefixes, suffixes, or compounding. German, for instance, can create novel words by smashing existing ones together. A system that only knows "wind" and "energy" might fail when it sees "Windenergie."

To solve this, Alexander Fraser and Marion Weller-Di Marco proposed modeling the internal structure of words. In Combining bilingual terminology mining and morphological modeling for domain adaptation in SMT, they demonstrated that if a system can't find a word in its dictionary, it should try to lemmatize it (find its root) or break it into components. Fabienne Cap and colleagues took this further in their WMT 2014 submission, using a "re-inflection" pipeline that translates into simplified forms and then synthetically generates the correct grammatical endings.

In the neural era, this evolved into Hybrid Word-Character Models. Minh-Thang Luong and Christopher D Manning showed in Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models that a system can operate mostly at the word level for efficiency, but "drop down" to a character-level recurrent neural network (RNN) whenever it encounters an unknown word. This allows the model to literally build a translation character-by-character for a word it has never seen.

Semantic Mapping in Vector Space

What if we don't have a paraphrase or a morphological root? We can use the "neighborhood" of the word in a vector space.

Pranava Madhyastha explored using Bilingual Embeddings to resolve OOVs. If the word "smartphone" is OOV, but the system knows that "smartphone" is semantically close to "telephone" in a monolingual English space, it can project that relationship into the target language's space to find a suitable candidate.

This is particularly useful for low-resource languages like Hindi or Bengali. Biman Gujral and Philipp Koehn showed in Translation of Unknown Words in Low Resource Languages that an ensemble approach—combining string similarity (for names), transliteration, and semantic mapping—can resolve up to half of all unknown words.

Mining Comparable Corpora

Finally, when the training data is the problem, researchers look to the web. Jianfeng Gao and Jian-Yun Nie pioneered techniques for Cross-Language Information Retrieval that involve mining "comparable corpora"—news articles in two languages that discuss the same event but aren't direct translations.

By identifying these parallel-ish sentences, as described by Dragos Stefan Munteanu and Daniel Marcu in Improved Machine Translation Performance via Parallel Sentence Extraction from Comparable Corpora, systems can discover translations for emerging entities and technical jargon in real-time, effectively expanding their own vocabulary as the world changes.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_resolving_out_of_vocabulary_words_in_mac_bf4f8eb2,
  title = {Resolving Out of Vocabulary Words in Machine Translation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/resolving-out-of-vocabulary-words-in-machine-translation/rdr_bf4f8eb2e66046d4b18f7e151aae39ac},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_resolving_out_of_vocabulary_words_in_mac_bf4f8eb2_v1,
  title = {Resolving Out of Vocabulary Words in Machine Translation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/resolving-out-of-vocabulary-words-in-machine-translation/rdr_bf4f8eb2e66046d4b18f7e151aae39ac/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}