Evaluating Cross Lingual Transfer for Morphological Analysis: a Case Study of Indian Languages

Siddhesh Pawar, Pushpak Bhattacharyya, Partha TalukdarSIGMORPHON 2023 · DOI

TL;DR

This paper explores how pre-trained multilingual language models can improve morphological analysis for low-resource Indian languages. By leveraging models like mT5, they demonstrate significant gains in accurately identifying grammatical features (like gender, number, and person) and extracting root words, even with limited language-specific data. This work is crucial for making NLP tools more inclusive and accessible for a wider range of languages.

What Problem Does This Paper Address?

Morphological analysis, the process of breaking down words into their constituent morphemes (like roots and affixes) and identifying their grammatical properties, is a foundational step in natural language processing (NLP). However, building accurate morphological analyzers for morphologically rich languages, especially those with limited annotated data (low-resource languages), is a significant challenge. Existing methods often rely on large, language-specific datasets or complex rule-based systems, which are expensive and time-consuming to develop. This paper addresses the gap of applying modern, powerful multilingual models to improve morphological analysis for a set of under-represented Indian languages, aiming to increase NLP inclusivity.

Methods

The core of this research lies in leveraging the Multilingual T5 (mT5) model, a powerful pre-trained text-to-text transformer that has learned shared representations across 101 languages. The authors investigate three modeling strategies for morphological analysis:

  • Joint Model: mT5 is used to predict both root words and morphological features (MSD tags) as a single, sequential output.
  • Multitask Model: mT5 is fine-tuned to perform MSD tagging and root word extraction as two distinct tasks, differentiated by input prefixes.
  • Independent Model: Separate mT5 models are trained for each task (MSD tagging and root word extraction).

They then evaluate these strategies in both monolingual (trained only on data from a single language) and multilingual (trained on data from multiple languages) settings. The primary focus is on cross-lingual transfer, where knowledge learned from high-resource languages is applied to low-resource languages. They specifically examine intra-family (e.g., Indo-Aryan to Indo-Aryan) and inter-family (e.g., Dravidian to Indo-Aryan) transfer.

Datasets

The study utilizes data from six Indian languages spanning two major families:

  • Dravidian Family: Tamil, Telugu, Kannada
  • Indo-Aryan Family: Bengali, Hindi, Marathi

For most languages (Bengali, Kannada, Hindi, Telugu), they used the UniMorph 3.0 dataset. For Tamil, they used data from the Tamil dependency treebank. For Marathi, they used a dataset from Bapat et al. (2010). The paper notes that the availability of annotated data varies significantly across these languages, with some having tens of thousands of examples and others only a few hundred.

Key Insights and Ideas

  • Multilingualism Boosts Performance: Pre-trained multilingual models like mT5 significantly improve morphological analysis accuracy for low-resource Indian languages, outperforming monolingual models. This is a strong validation of the "multilinguality hypothesis."
  • Transfer is Effective Across Language Families: Cross-lingual transfer is not limited to languages within the same family; models can effectively learn from Dravidian languages to improve analysis for Indo-Aryan languages, and vice-versa.
  • Low-Resource Scenarios Benefit Most: The gains from multilingual transfer are most pronounced when the target language has very little annotated data.
  • Task-Specific Modeling Matters: Training separate models for root word extraction and MSD tagging (Independent Model) generally yields the best performance, outperforming joint or multitask approaches.
  • Copy Bias is Key for Root Extraction: The "copy bias" in models, where they tend to copy parts of the input to the output, plays a crucial role in root word extraction, as the output often shares many characters with the input surface form.
  • Gender Tagging is Particularly Challenging: Gender assignment, especially for nouns, can be arbitrary and often requires external knowledge (like dictionaries) or contextual information, which is not always available in the datasets.

Key Results

  • Multilingual models achieved an average 7% increase in GNP tagging accuracy compared to monolingual settings.
  • For Dravidian languages like Tamil and Kannada, multilingual models showed an average 7.8% increase in overall accuracy.
  • In low-resource scenarios (e.g., 1000 target language examples), multilingual models showed substantial improvements, with accuracy increases of up to 54% for Bengali and 33% for Kannada for MSD tagging.
  • For root word extraction, the independent model strategy achieved the highest accuracies, with multilingual models outperforming monolingual ones. For example, in Marathi, the multilingual independent model achieved 95.2% accuracy for root word extraction, compared to 78.2% for the monolingual independent model.

Limitations and Future Work

  • Contextual Data: The current work relies on decontextualized data. The absence of contextual information limits the model's ability to disambiguate certain tags, particularly for languages where morphology is expressed through word order or phrases (e.g., Hindi, Bengali). Future work could explore incorporating contextual information.
  • Phrase-Based Morphology: For highly agglutinative languages, morphological markings can span multiple words. The current token-level analysis might not fully capture this.
  • Dataset Availability: The study is limited by the availability of annotated data for specific parts of speech and features across languages. The authors suggest that future research could focus on creating more comprehensive datasets.
  • Model Availability: The availability of pre-trained multilingual models for all desired languages is a constraint.
  • Future Directions: The authors envision extending their framework to handle more tags, more low-resource languages as data becomes available, and applying this to bias-aware machine translation. They also suggest further investigation into the role of historical contact between languages in cross-lingual transfer.

Summary updated

BibTeX
@inproceedings{pawar2023evaluating,
  author = {Siddhesh Pawar and Pushpak Bhattacharyya and Partha Talukdar},
  title = {{Evaluating Cross Lingual Transfer for Morphological Analysis: a Case Study of Indian Languages}},
  booktitle = {SIGMORPHON},
  year = {2023},
  doi = {10.18653/v1/2023.sigmorphon-1.3},
}
Figure