Bridging the Orthographic Gap in Multilingual Transfer

489 concepts315 papers1 proposal

Modern multilingual models often appear to possess a "universal" understanding of language, yet this universality frequently breaks down at the boundary of the writing system. When a model trained on Latin-script English is asked to generalize to Hindi in Devanagari or Russian in Cyrillic, it encounters an orthographic barrier. This divergence in scripts creates a fundamental disconnect in how models represent and transfer linguistic knowledge.

The Orthographic Transfer Gap

The primary reason multilingual models struggle with different scripts is their reliance on shared subword or character-level representations. As Yang et al. note, transfer mechanisms that utilize character-level features are effectively limited to language pairs that share a common alphabet. Without this overlap, the model cannot recognize morphological similarities or shared lexical roots (cognates) that would otherwise facilitate knowledge transfer.

This dependency creates a "transfer gap." Lewis et al. demonstrated that even state-of-the-art models perform significantly worse on languages like Arabic and Hindi compared to Latin-script languages, despite being trained on massive multilingual corpora. Similarly, Biten et al. found that zero-shot transfer in visual reasoning tasks is severely hindered when the target language utilizes a different script from the training data, suggesting that shared orthography is a critical anchor for cross-lingual generalization.

Script Isolation and Information Silos

When languages do not share a writing system, they often become "information silos" within the model. Sorokin et al. observed that languages with unique scripts exhibit high "self-usage" rates, meaning the model struggles to bridge knowledge between distinct scripts and instead relies solely on limited native-language resources. This orthographic isolation prevents the effective flow of information from high-resource languages (like English) to low-resource languages that use non-overlapping character sets, such as Cherokee (ChrEn) or various Turkic languages (Mirzakhalov et al.).

Interestingly, this isolation can have unexpected side effects. Zhang et al. found that catastrophic forgetting—where a model loses old knowledge while learning new tasks—is actually reduced when the new tasks involve different scripts. The distinct writing systems allow the model to utilize separate regions of its embedding space, preventing the erasure of previously learned Latin-script knowledge when learning a Cyrillic-script task.

Bridging the Gap through Phonology

To overcome the script barrier, researchers have proposed moving beyond how words are written to how they sound. Since related languages often share phonetic roots even if their scripts differ, phonology provides a "universal" bridge.

Bharadwaj et al. proposed using phonological representations to identify similar-sounding entities across different scripts for Named Entity Recognition. By integrating phonemic transcriptions (such as the International Phonetic Alphabet) as a second modality, the authors of PhoneXL successfully bridged the gap between Chinese, Japanese, Korean, and Vietnamese (CJKV) languages. Tools like PanPhon, introduced by Mortensen et al., facilitate this by mapping IPA segments to articulatory feature vectors, allowing models to operate in a shared phonological space regardless of the underlying orthography.

Transliteration and Script Adaptation

Another strategy involves explicitly mapping divergent scripts into a common format. Parikh and Gautam et al. highlight the importance of transliteration, particularly for "Romanized" versions of South Asian languages (like Hinglish). Models pre-trained on native scripts often fail to recognize the same language when written in Latin characters; back-transliterating this text into the native script allows the model to leverage its existing linguistic knowledge.

For languages with entirely "unseen" scripts that were absent during pre-training, standard embedding adjustments are often insufficient. Pfeiffer et al. addressed this by developing data-efficient adaptation methods, such as MAD-X 2.0, which use adapters and matrix factorization to help models learn new scripts without encountering a sea of "Unknown" ([UNK]) tokens.

Ultimately, as Ponti et al. argue, the goal is to decouple linguistic priors from orthographic encoding. By moving toward representations that are script-invariant, researchers aim to create models that can truly generalize across the diverse writing systems of the world's languages.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 328 papers
  2. Version 1 · 2026-10-04 · 329 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_orthographic_divergence_63727977,
  title = {orthographic divergence},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/orthographic-divergence/rdr_63727977f1e9436288faf8c294028d58},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_orthographic_divergence_63727977_v2,
  title = {orthographic divergence},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/orthographic-divergence/rdr_63727977f1e9436288faf8c294028d58/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake