TL;DR
This paper introduces "cognate projection" as a novel method to improve low-resource inflection generation by leveraging high-resource language data. By projecting cognates (words with shared linguistic origins) from a high-resource language to a low-resource one, they create synthetic training data that significantly boosts performance, especially when no low-resource training data is available.
What Problem Does This Paper Address?
Generating correct inflected word forms for languages with limited linguistic data (low-resource languages) is a significant challenge. Traditional methods often rely on large amounts of language-specific training data, which is scarce for many languages. This paper tackles the problem of inflection generation in low-resource settings by exploring cross-lingual transfer techniques, specifically aiming to make the most of available high-resource language data. The core issue is how to effectively transfer morphological knowledge from a well-resourced language to a poorly-resourced one when direct training data is minimal.
Methods
The authors propose two main approaches for cross-lingual transfer using cognate projection:
- Data Projection: This method involves projecting the entire high-resource (HR) training dataset (lemmas and their inflected forms) into the low-resource (LR) language. This creates a large, synthetic LR training set. The underlying assumption is that HR inflection patterns can be mirrored in the corresponding LR patterns, especially for closely related languages.
- Instance Projection: This is a more complex, three-step process. First, an individual LR test instance (lemma + tags) is projected into the HR language. Second, this projected instance is inflected using a model trained on HR data. Finally, the resulting HR inflected form is projected back into the LR language. This approach performs inflection in the HR language and aims to see if higher HR accuracy can compensate for projection errors.
Both methods leverage DTLM (Discriminative Transduction with Language Models) for cognate projection and inflection, and OpenNMT for high-resource inflection generation. DTLM is a powerful sequence transduction model that integrates character and word language models and has shown strong performance in low-data scenarios.
Datasets
The paper utilizes data from the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology. The language pairs tested are:
- Kashubian-Polish (closely related, sometimes considered dialects)
- Occitan-Spanish (share many morphological features)
- Romanian-Latin (evolved from Latin)
- Hindi-Bengali (related but use different scripts)
External resources used include:
- UniMorph: For target word lists for DTLM.
- Wikipedia: For target word lists for DTLM.
- OPUS: For parallel bitexts (software documentation) for Polish-Kashubian and Spanish-Occitan.
- OpenSubtitles (v2018): For parallel bitexts for Hindi-Bengali.
- A parallel corpus of Bible translations in 100 languages: For Romanian-Latin.
The paper also mentions using FAST_ALIGN for bitext alignment and uroman for romanization.
Key Insights and Ideas
- Cognate Projection as a Powerful Transfer Mechanism: The central idea is that cognates, by their very nature, carry shared morphological and phonological information. Projecting these cognates allows us to bridge the gap between languages, effectively creating "virtual" training data for low-resource languages.
- Leveraging HR Data is Crucial for Low-Resource Inflection: The results strongly suggest that even with imperfect projection, utilizing high-resource data is significantly better than relying solely on minimal or no low-resource data.
- Data Projection Outperforms Instance Projection in Some Cases: While Instance Projection is more intricate, Data Projection, which projects the entire HR dataset, yielded better results for some language pairs, highlighting the benefit of a larger synthetic training set.
- The Importance of Target Wordlists: Augmenting DTLM models with target wordlists extracted from monolingual corpora demonstrably improved inflection accuracy, especially for less closely related languages.
Key Results
The paper reports significant improvements using cognate projection, particularly in scenarios with no low-resource training data:
- Data Projection (System 3) and Instance Projection (System 4) significantly outperform baselines when no low-resource training data is available. For instance, for Polish-Kashubian, cognate projection (System 3) achieved a Word Accuracy of 66.0%, compared to a Copy Baseline of 12.0% and No Projection (DTLM on HR data only) of 16.0%.
- Augmenting LR training data with projected HR data did not consistently improve accuracy in most cases, suggesting that the projection itself is the primary driver of improvement when LR data is scarce.
- The Polish-Kashubian results are highlighted as a proof of concept, demonstrating that cognate projection can outperform baselines even when only HR data is used, suggesting its potential for new research directions.
Limitations and Future Work
- Accuracy of Projection: The accuracy of cognate projection itself can be a bottleneck. The paper notes that the Romanian-Latin projection accuracy was relatively low, potentially due to the specific domain of the Bible corpus used.
- Typological Differences: Significant typological differences between languages can pose challenges, even with cognate projection. The paper mentions that Latin nominal inflection paradigms, with their case systems, differ greatly from Romanian.
- Tag Order Inconsistencies: Differences in the order of morphological tags between languages can also complicate transfer. While the authors did not address this directly in the current task, they refer to prior work for principled solutions.
- Future Work: The authors suggest that their Polish-Kashubian results are a strong indicator for further research into cognate projection for inflection generation. They also implicitly point towards improving the projection models themselves and handling more complex linguistic differences.
Summary updated