Out of Vocabulary Word Handling for Machine Translation

45 concepts42 papers1 proposal

Natural language is an open, evolving system. New words emerge constantly through cultural shifts, technical innovations, and dialectal variations. Traditional machine translation (MT) systems, however, operate on a closed vocabulary—a fixed dictionary or token set learned during training. When an MT system encounters an out-of-vocabulary (OOV) or unknown word (UNK), the translation pipeline can stall or degrade significantly.

Historically, early translation systems relied on rudimentary fallback routines. For instance, the system evaluated by J. McDaniel et al. in An evaluation of the usefulness of machine translations produced at the National Physical Laboratory, Teddington, with a summary of the translation methods used basic "anglicizing" routines to handle non-dictionary words. Early rule-based architectures, such as the METAL system detailed by Geert Adriaens et al. in The Self-Extending Lexicon: Off-Line and On-Line Defaulting of Lexical Information in the METAL Machine Translation System, introduced "defaulting" modules to automatically extend lexicons either off-line during batch processing or on-line during real-time translation. Without these mechanisms, systems often copy the source word verbatim or omit it entirely, leading to severe information loss.

To solve the OOV problem, modern NLP relies on a mix of structural deconstruction, semantic substitution, phonetic transliteration, and real-time adaptation.

Subword and Character-Level Deconstruction

If we treat words as indivisible, atomic units, any unseen word becomes an impenetrable wall. However, words are fundamentally compositional, built from smaller morphemes and characters. By shifting the grain size of translation from whole words to subwords or characters, models can synthesize translations for unseen words based on familiar building blocks.

This intuition underpins cascaded and hybrid translation models. Jörg Tiedemann demonstrated in Character-Based PSMT for Closely Related Languages that inserting a character-based translation step specifically for OOV words before applying a standard word-based model significantly reduces untranslated source text when translating between closely related languages. Similarly, Nadir Durrani et al. in Edinburgh's Phrase-based Machine Translation Systems for WMT-14 utilized unsupervised character-based models to provide a robust fallback mechanism when direct lexical translations were missing.

In modern neural architectures, this concept is formalized through subword tokenization algorithms like Byte-Pair Encoding (BPE). For example, Himanshu Choudhary et al. in Neural Machine Translation for English-Tamil combined pre-trained word embeddings with BPE to mitigate the OOV problem in morphologically rich languages where word variations are frequent. When standard Transformer models struggle with unseen words, leveraging subword information—as shown by Rohit Vp et al. in Cyber Protectors@DravidianLangTech 2025: Abusive Tamil and Malayalam Text Targeting Women on Social Media using FastText—allows the system to capture the underlying morphological structure of novel vocabulary.

Morphological and Semantic Substitution

When a word cannot be broken down into recognizable subwords, another strategy is to transform the unknown into the known before it reaches the core translation engine. This can be achieved by analyzing the word's structural roots or by leveraging its surrounding context.

Bruno Cartoni in Lexical Morphology in Machine Translation: A Feasibility Study explored using lexical morphology to analyze and translate neologisms based on their internal structural components. Alternatively, if the exact word is missing from the vocabulary, the system can substitute it with a known synonym that shares the same semantic space. Ping Wang in A study of an intelligent algorithm combining semantic environments for the translation of complex English sentences proposed a pre-processing algorithm that leverages WordNet and semantic context to replace OOV terms with known synonyms prior to translation, ensuring the model processes a complete semantic representation rather than a generic placeholder.

[Source Text with OOV] ---> [Contextual Analysis] ---> [Synonym Substitution] ---> [Standard MT Engine]

This substitution strategy is also highly effective for dialectal translation, where variations often prevent direct matching. In Improving Egyptian-to-English SMT by Mapping Egyptian into MSA, Abraham Ittycheriah et al. addressed OOV words in dialectal Arabic by applying spelling corrections, context-aware synonyms, and dialect-specific rules to map informal words into Modern Standard Arabic (MSA) equivalents before translation.

Cross-Lingual Transliteration for Proper Names

A large portion of OOV words consists of proper nouns, technical terms, and loanwords. These words do not have a semantic translation; instead, they must be converted from the source alphabet to the target alphabet while preserving their pronunciation.

As noted by Colin Cherry et al. in Discriminative Substring Decoding for Transliteration, standard machine translation systems frequently fail when encountering loanwords, often copying or deleting them. To preserve phonetic identity across different writing systems, researchers developed direct script-to-script mapping models. Asif Ekbal et al. in A Modified Joint Source-Channel Model for Transliteration bypassed complex phonetic intermediates in favor of direct character mapping between scripts.

To handle the messy alignments that occur between structurally distinct scripts, Yan Shao et al. in Boosting English-Chinese Machine Transliteration via High Quality Alignment and Multilingual Resources utilized a custom Expectation-Maximization (EM) algorithm to refine character alignments and leveraged multilingual dictionaries to boost accuracy. For low-resource language pairs where training data is sparse, the authors of Low-Resource Machine Transliteration Using Recurrent Neural Networks of Asian Languages combined recurrent neural networks (RNNs) with alignment representations and pre-trained embeddings to accurately transliterate unseen terms. Even within the same language family, converting between different scripts requires sophisticated handling of missing vowels and word segmentations, as demonstrated by Gurpreet Singh Lehal et al. in Conversion between Scripts of Punjabi: Beyond Simple Transliteration.

Dynamic and Real-Time Adaptation

Static training sets cannot predict the future. When new named entities—such as rising public figures, trending products, or borrowed words—emerge, offline vocabularies become obsolete. To solve this, translation systems must adapt dynamically to the linguistic environment.

One approach is to leverage the internet as an external, real-time knowledge base. Lei Cui et al. in Machine Translation with Real-Time Web Search introduced a framework that treats MT as a real-time web search task, querying search engines on the fly to discover translation candidates and build localized language models from snippets to resolve fresh OOV terms.

[Unseen Entity Encountered] ---> [Live Web Search Query] ---> [Extract Snippets] ---> [Dynamic Translation Candidate]

When human corrections are available but sparse, systems can use few-shot learning to rapidly ingest new terms. Farid Arthaud et al. in Few-shot learning through contextual data augmentation proposed a framework using contextual data augmentation to adapt MT models to unseen words with minimal human examples. Additionally, systems can dynamically extract translation pairs directly from parallel corpora to build and update MT dictionaries on the fly, as shown by Akira Kumano et al. in Building an MT Dictionary From Parallel Texts Based on Linguistic and Statistical Information. By modeling how languages borrow words from one another over time, as explored by Chris Dyer et al. in Cross-Lingual Bridges with Models of Lexical Borrowing, translation systems can build predictive bridges that handle low-resource OOV words before they break the pipeline.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_out_of_vocabulary_word_handling_out_of_v_d00a242f,
  title = {out-of-vocabulary word handling (out-of-vocabulary words)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/out-of-vocabulary-word-handling-out-of-vocabulary-words/rdr_d00a242f694142bf9a1411caa98c5c6e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_out_of_vocabulary_word_handling_out_of_v_d00a242f_v1,
  title = {out-of-vocabulary word handling (out-of-vocabulary words)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/out-of-vocabulary-word-handling-out-of-vocabulary-words/rdr_d00a242f694142bf9a1411caa98c5c6e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake