TL;DR
This paper introduces a novel computational model for understanding and predicting lexical borrowing between languages, leveraging linguistic theories like Optimality Theory. This model significantly improves machine translation for low-resource languages by identifying and translating out-of-vocabulary words that have been borrowed.
What Problem Does This Paper Address?
Modern Natural Language Processing (NLP) tools heavily rely on language-specific data, which is abundant for high-resource languages but scarce for the vast majority of the world's languages. Existing methods for bridging this gap, such as using parallel corpora for cross-lingual projection, are expensive and require large amounts of data. This paper addresses the challenge of creating NLP resources for low-resource languages by exploiting the phenomenon of linguistic borrowing. Borrowing, the adoption of words from one language into another, is pervasive and can act as a natural bridge between resource-rich and resource-limited languages. The core problem is to computationally identify these borrowing relationships and use them to improve NLP tasks, particularly machine translation, by handling out-of-vocabulary (OOV) words.
Methods
The authors propose a computational model for lexical borrowing that is conceptually divided into three parts:
- Phonetic Conversion: Orthographic word forms are converted into their International Phonetic Alphabet (IPA) representations.
- Candidate Generation: Potential loanword pronunciations are generated from a donor word. This involves rule-based transformations like syllabification, insertion, deletion, and substitution of phonemes and morphemes.
- Constraint-Based Ranking: Generated loanword candidates are ranked using a model inspired by Optimality Theory (OT). OT posits that language-specific constraints govern how words are adapted. The model uses a weighted sum of constraint violations, where weights are learned from a small set of donor-recipient pairs. This approach aims to be more data-efficient than linguistically naive baselines.
The model is implemented as a cascade of finite-state transducers. For machine translation applications, the borrowing model is used to identify plausible donor words for OOV words in a low-resource language. These donor words are then translated using a resource-rich language's machine translation system, creating "synthetic phrases" that are integrated into the low-resource MT system. Phonetic and semantic features are used to assess the plausibility of these borrowed word translations.
Datasets
- Arabic-English and Swahili-English: Bitexts were used to extract training data for the borrowing model.
- Italian-Maltese: A parallel corpus from the EUbookshop corpus was used.
- French-Romanian: Transcribed TED talks were used to simulate a resource-poor scenario.
- Helsinki Corpus of Swahili (HCS): Used for manual error analysis of OOV words.
- Global Voices project: Used for Swahili-English parallel corpus.
- OPUS collection: Used for Maltese-English parallel corpus.
- Transcribed TED talks: Used for Romanian-English parallel corpus.
The paper also mentions using pronunciation dictionaries (e.g., CMU Arabic, GLOBALPHONE for French) and word embeddings trained on large monolingual corpora.
Key Insights and Ideas
- Linguistic Borrowing as a Bridge: The core idea is that lexical borrowing, a natural linguistic phenomenon, can serve as a powerful and data-efficient bridge for cross-lingual NLP tasks, especially for low-resource languages.
- Optimality Theory for Data Efficiency: Applying OT constraints allows the model to learn linguistic adaptation rules with significantly less training data compared to purely data-driven or linguistically naive methods. This is crucial for low-resource scenarios.
- Handling OOV Words in MT: The model provides a practical solution for improving machine translation by identifying and translating out-of-vocabulary words that are loanwords, which are often regular content words in low-resource languages.
- Systematic Phonological Adaptation: The model captures the systematic, rule-governed nature of phonological changes that occur when words are borrowed, which is key to accurate prediction.
Key Results
- Intrinsic Evaluation: The borrowing model achieved high reachability (percentage of donor-recipient pairs that are reachable) and significantly reduced ambiguity (average number of outputs per input) compared to naive models.
- Accuracy: The learned OT constraint weights model achieved significantly higher accuracy (up to 48.4% for Arabic-Swahili) compared to orthographic, phonetic, and transliteration baselines.
- Machine Translation Improvement:
- Swahili-English MT saw improvements of up to +1.6 BLEU by augmenting with translated OOV loanwords.
- Maltese-English MT improved by up to +0.8 BLEU.
- Romanian-English MT showed small but significant improvements.
- Error Analysis: The main sources of errors were identified as:
- Reachability issues due to complex Swahili morphology.
- Loanword production errors from the borrowing system itself.
- Arabic-English translation errors (often OOV Arabic words).
- Swahili-English translation errors (less frequent).
Limitations and Future Work
- Morphological Complexity: The model struggles with the complex morphology of languages like Swahili, which can lead to unreachable paths for loanword candidates.
- Semantic Shift: The model does not explicitly handle historical changes in word meaning.
- Future Work: The authors suggest exploring:
- Monolingually identifying borrowed words.
- Automatically identifying donor languages or their phonological properties.
- Jointly modeling borrowing from multiple sources.
- Reducing language-specific engineering effort.
- Integrating borrowing knowledge into other downstream NLP applications.
Summary updated