Cross-Lingual Bridges with Models of Lexical Borrowing

Yulia Tsvetkov, Chris DyerJAIR 2016 · DOI

TL;DR

This paper introduces a novel computational model for understanding and predicting lexical borrowing between languages, leveraging linguistic theories like Optimality Theory. This model significantly improves machine translation for low-resource languages by identifying and translating out-of-vocabulary words that have been borrowed.

What Problem Does This Paper Address?

Modern Natural Language Processing (NLP) tools heavily rely on language-specific data, which is abundant for high-resource languages but scarce for the vast majority of the world's languages. Existing methods for bridging this gap, such as using parallel corpora for cross-lingual projection, are expensive and require large amounts of data. This paper addresses the challenge of creating NLP resources for low-resource languages by exploiting the phenomenon of linguistic borrowing. Borrowing, the adoption of words from one language into another, is pervasive and can act as a natural bridge between resource-rich and resource-limited languages. The core problem is to computationally identify these borrowing relationships and use them to improve NLP tasks, particularly machine translation, by handling out-of-vocabulary (OOV) words.

Methods

The authors propose a computational model for lexical borrowing that is conceptually divided into three parts:

  1. Phonetic Conversion: Orthographic word forms are converted into their International Phonetic Alphabet (IPA) representations.
  2. Candidate Generation: Potential loanword pronunciations are generated from a donor word. This involves rule-based transformations like syllabification, insertion, deletion, and substitution of phonemes and morphemes.
  3. Constraint-Based Ranking: Generated loanword candidates are ranked using a model inspired by Optimality Theory (OT). OT posits that language-specific constraints govern how words are adapted. The model uses a weighted sum of constraint violations, where weights are learned from a small set of donor-recipient pairs. This approach aims to be more data-efficient than linguistically naive baselines.

The model is implemented as a cascade of finite-state transducers. For machine translation applications, the borrowing model is used to identify plausible donor words for OOV words in a low-resource language. These donor words are then translated using a resource-rich language's machine translation system, creating "synthetic phrases" that are integrated into the low-resource MT system. Phonetic and semantic features are used to assess the plausibility of these borrowed word translations.

Datasets

  • Arabic-English and Swahili-English: Bitexts were used to extract training data for the borrowing model.
  • Italian-Maltese: A parallel corpus from the EUbookshop corpus was used.
  • French-Romanian: Transcribed TED talks were used to simulate a resource-poor scenario.
  • Helsinki Corpus of Swahili (HCS): Used for manual error analysis of OOV words.
  • Global Voices project: Used for Swahili-English parallel corpus.
  • OPUS collection: Used for Maltese-English parallel corpus.
  • Transcribed TED talks: Used for Romanian-English parallel corpus.

The paper also mentions using pronunciation dictionaries (e.g., CMU Arabic, GLOBALPHONE for French) and word embeddings trained on large monolingual corpora.

Key Insights and Ideas

  • Linguistic Borrowing as a Bridge: The core idea is that lexical borrowing, a natural linguistic phenomenon, can serve as a powerful and data-efficient bridge for cross-lingual NLP tasks, especially for low-resource languages.
  • Optimality Theory for Data Efficiency: Applying OT constraints allows the model to learn linguistic adaptation rules with significantly less training data compared to purely data-driven or linguistically naive methods. This is crucial for low-resource scenarios.
  • Handling OOV Words in MT: The model provides a practical solution for improving machine translation by identifying and translating out-of-vocabulary words that are loanwords, which are often regular content words in low-resource languages.
  • Systematic Phonological Adaptation: The model captures the systematic, rule-governed nature of phonological changes that occur when words are borrowed, which is key to accurate prediction.

Key Results

  • Intrinsic Evaluation: The borrowing model achieved high reachability (percentage of donor-recipient pairs that are reachable) and significantly reduced ambiguity (average number of outputs per input) compared to naive models.
  • Accuracy: The learned OT constraint weights model achieved significantly higher accuracy (up to 48.4% for Arabic-Swahili) compared to orthographic, phonetic, and transliteration baselines.
  • Machine Translation Improvement:
    • Swahili-English MT saw improvements of up to +1.6 BLEU by augmenting with translated OOV loanwords.
  • Maltese-English MT improved by up to +0.8 BLEU.
  • Romanian-English MT showed small but significant improvements.
  • Error Analysis: The main sources of errors were identified as:
  • Reachability issues due to complex Swahili morphology.
  • Loanword production errors from the borrowing system itself.
  • Arabic-English translation errors (often OOV Arabic words).
  • Swahili-English translation errors (less frequent).

Limitations and Future Work

  • Morphological Complexity: The model struggles with the complex morphology of languages like Swahili, which can lead to unreachable paths for loanword candidates.
  • Semantic Shift: The model does not explicitly handle historical changes in word meaning.
  • Future Work: The authors suggest exploring:
  • Monolingually identifying borrowed words.
  • Automatically identifying donor languages or their phonological properties.
  • Jointly modeling borrowing from multiple sources.
  • Reducing language-specific engineering effort.
  • Integrating borrowing knowledge into other downstream NLP applications.

Summary updated

BibTeX
@article{tsvetkov2016cross,
  author = {Yulia Tsvetkov and Chris Dyer},
  title = {{Cross-Lingual Bridges with Models of Lexical Borrowing}},
  journal = {J. Artif. Intell. Res.},
  year = {2016},
  doi = {10.1613/jair.4786},
}
Figure