High Accuracy Machine Transliteration Across Diverse Scripts

27 concepts25 papers1 proposal

Machine transliteration is the process of converting text from one writing system to another while preserving the phonetic structure of the original words. Unlike translation, which focuses on meaning, transliteration focuses on sound. This is particularly critical for proper nouns, technical terms, and loanwords that do not exist in standard bilingual dictionaries.

Recent research has moved beyond simple character-to-character mapping, achieving state-of-the-art performance by integrating phonetic intuition, contextual disambiguation, and advanced neural architectures.

Beyond Simple Character Mapping

The primary challenge in transliteration is that scripts rarely have a one-to-one correspondence. A single sound in a source language might be represented by multiple characters in the target script, or vice versa.

To address this, Kang et al. and Kim proposed using multiple unbounded overlapping phoneme chunks for English-to-Korean transliteration. By looking at chunks of sound rather than isolated letters, their system reached accuracies over 90%, as it could better capture the phonetic nuances of how English vowels and consonants transform into Korean characters. Similarly, Pasternack et al. and Roth demonstrated that treating transliteration as a flexible process of segmenting source words into variable-length substrings significantly outperforms rigid models, yielding a 10% accuracy improvement across Chinese, Hebrew, and Russian.

For languages with complex script relationships, such as Punjabi (which uses Gurmukhi in India and Shahmukhi in Pakistan), simple mapping is insufficient due to missing vowels and word segmentation differences. Lehal et al. and Saini developed hybrid approaches that address these linguistic hurdles, achieving word-level accuracies exceeding 98%.

Leveraging Context and Semantics

Transliteration is often ambiguous; a single phonetic string in Japanese Katakana could represent several different English words. To resolve this, researchers have integrated higher-level linguistic information.

Abe et al. et al. showed that combining morphological, syntactic, and semantic analysis allows a system to achieve 90.5% accuracy on non-segmented Japanese text. By understanding the grammatical role of a word, the system can better predict its correct script representation. In the realm of back-transliteration (e.g., Katakana back to English), Goto et al. et al. demonstrated that leveraging the "target English context" helps disambiguate phonetic similarities, significantly improving the accuracy of generating personal names.

The Importance of Word Origin and Loanwords

The rules of transliteration often change based on where a word comes from. A technical term borrowed from English into Korean follows different phonetic patterns than a native Korean word.

Oh et al. and Choi found that incorporating the origin of a word into the model led to a 31% increase in accuracy. This suggests that the model "learns" different transformation rules for different etymological paths.

In some cases, the best transliteration strategy is to avoid it for certain words. Bisazza et al. et al. discovered that in Arabizi-to-English translation pipelines, automated systems can actually outperform humans by not transliterating loanwords. If a Romanized Arabic text contains an English loanword, leaving it in its original Roman form allows downstream translation models to recognize it instantly, whereas a human's attempt to force it into Arabic script might introduce errors.

Neural Architectures and Low-Resource Scenarios

As the field has evolved, probabilistic models like Hidden Markov Models (HMMs) and Weighted Finite State Transducers (WFSTs) have given way to neural approaches. Varadarajan et al. and Rao initially pushed the boundaries of HMMs by introducing "e-extensions" (null characters) to handle script misalignments.

More recently, Recurrent Neural Networks (RNNs) and Large Language Models (LLMs) have set new benchmarks, especially for low-resource languages or informal text. Le et al. and Sadat utilized RNNs with pre-trained embeddings to improve English-to-Vietnamese transliteration for out-of-vocabulary words. For informal social media text, Kumar et al. et al. fine-tuned LLaMa 3.1 to convert Romanized Hindi back to Devanagari, significantly reducing error rates by leveraging the model's broad linguistic pre-training.

These advancements ensure that transliteration remains a robust bridge for cross-lingual information retrieval, sentiment analysis, and the preservation of linguistic heritage, as seen in the work by Naira et al. et al. on the Comorian Kamar-Eddine script.

Key Foundational Work

An Automatic Translation System of Non-Segmented Kana Sentences Into Kanji-Kana Sentences (1980) — contributed 1 concept elements to this direction.
A Kana-Kanji Translation System for Non-Segmented Input Sentences Based on Syntactic and Semantic Analysis (1986) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_state_of_the_art_transliteration_perform_c0448271,
  title = {state-of-the-art transliteration performance},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/state-of-the-art-transliteration-performance/rdr_c044827182a54f99a64d36aaf95b706f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_state_of_the_art_transliteration_perform_c0448271_v1,
  title = {state-of-the-art transliteration performance},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/state-of-the-art-transliteration-performance/rdr_c044827182a54f99a64d36aaf95b706f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake