The CMU-LTI submission to the SIGMORPHON 2020 Shared Task 0: Language-Specific Cross-Lingual Transfer

Nikitha Murikinati, Antonios AnastasopoulosACL 2020

TL;DR

This paper presents the CMU-LTI submission to the SIGMORPHON 2020 Shared Task, focusing on language-specific cross-lingual transfer for morphological inflection. Their approach, adapted from previous work, achieved competitive results by strategically selecting transfer languages based on language characteristics and data availability, even without hyperparameter tuning.

What Problem Does This Paper Address?

Morphological inflection, the process of changing a word's form to convey grammatical information (like tense or number), is a fundamental aspect of language. Computationally, this task involves mapping a word's base form (lemma) and a set of morphological tags to its inflected form. The SIGMORPHON 2020 Shared Task specifically challenged systems to generalize across typologically diverse languages, including low-resource ones, without relying on large amounts of language-specific data. This is a significant challenge because many languages have complex inflectional systems, and building separate models for each is impractical, especially for languages with limited digital resources.

Methods

The CMU-LTI submission builds upon their previous winning system, which utilized a neural multi-source encoder-decoder architecture. Key to their approach is cross-lingual transfer, where knowledge learned from a high-resource "source" language is applied to a low-resource "target" language.

Their novel contribution lies in the language-specific transfer regimes. Instead of a one-size-fits-all approach, they strategically chose how to leverage cross-lingual transfer for each of the 90 test languages:

  • No transfer: For high-resource languages.
  • Single related high-resource language: For some languages.
  • Multiple related languages: For others.

They also employed data hallucination, a technique that artificially augments training data by introducing random character substitutions in stem-like regions. This was particularly adapted for tonal languages to avoid corrupting tone diacritics. For languages with different scripts, they experimented with transliterating transfer languages into the target language's script or romanizing the target language's training data.

Crucially, the authors state they did not tune any hyperparameters, using default settings optimized for extremely low-resource scenarios.

Datasets

The paper uses data from the SIGMORPHON 2020 Shared Task 0, which comprises morphological inflection data for 90 typologically diverse languages. The task involves mapping lemmas and morphological tags to inflected forms. The authors also mention using data from related languages for cross-lingual transfer and augmenting training data through hallucination.

Key Insights and Ideas

  • Strategic cross-lingual transfer is key: The success of their system, even without tuning, highlights the power of carefully selecting source languages for transfer, rather than blindly applying it. This suggests that linguistic relatedness and data availability are critical factors.
  • Default parameters can be competitive: The fact that their system, using default hyperparameters tuned for low-resource settings, achieved strong results indicates that the core architecture and transfer strategy are robust. This challenges the assumption that extensive tuning is always necessary for state-of-the-art performance.
  • Data hallucination needs careful adaptation: While generally useful, the authors found that naive data hallucination can be detrimental, especially for languages with complex features like tone or agglutination, necessitating script-aware or feature-aware adaptation.
  • Script differences are a hurdle: The paper reinforces the observation that cross-lingual transfer is less effective when source and target languages use different scripts, motivating their work on transliteration and romanization.

Key Results

The CMU-LTI system achieved a macro-averaged accuracy of 80.65% across all 90 languages, ranking 20th out of 31 systems. Despite this overall ranking, their system was statistically tied for the best system in 25 of the 90 languages. They highlight competitive performance on languages like Ludian (lud), Võro (vro), and Middle Low German (gml), attributing this to their informed selection of transfer languages. Conversely, they performed poorly on Algic (Cree) and Tungusic (Evenki), suspecting issues with their data hallucination technique for these highly complex languages.

Limitations and Future Work

The authors acknowledge that their system's performance could be improved with hyperparameter tuning, especially for higher-resource settings. They also suspect that their data hallucination technique might not be suitable for languages with very complex morphological structures (like Cree) or specific phonological features (like vowel harmony in Evenki).

Future work directions include:

  • Investigating automatic suggestion systems for selecting transfer languages.
  • Updating their model to use sparsemax for potentially better exact search capabilities.
  • Employing language/task selection methods to tune systems on small subsets of languages that generalize well.
  • Utilizing differential data selection for optimizing multilingual models across multiple objectives.

Summary updated

BibTeX
@inproceedings{murikinati2020cmu,
  author = {Nikitha Murikinati and Antonios Anastasopoulos},
  title = {{The CMU-LTI submission to the SIGMORPHON 2020 Shared Task 0: Language-Specific Cross-Lingual Transfer}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
}
Figure