Part of Speech Tagging in Luyia: A Bantu Macrolanguage

Kenneth SteimelCOLING 2018

TL;DR

This paper explores how to perform Part-of-Speech (POS) tagging for Luyia, a Bantu macrolanguage in Kenya, by leveraging data from related Luyia varieties. Surprisingly, a more distantly related variety (Tiriki) proved more effective than a closer one (Bukusu) for tagging Wanga, suggesting that vocabulary overlap and biases in the training data can outweigh linguistic proximity.

What Problem Does This Paper Address?

Part-of-Speech (POS) tagging, the task of assigning grammatical categories (like noun, verb, adjective) to words in a text, is a fundamental step in Natural Language Processing (NLP). While POS taggers are well-established for high-resource languages like English, they remain a significant challenge for morphologically rich, low-resource languages, particularly those in the Bantu language family. Luyia, a macrolanguage spoken by millions in Kenya, is a prime example. Its complex morphology makes traditional rule-based approaches difficult and expensive to develop, especially given the ongoing documentation of these languages. This paper aims to address this gap by investigating how to build effective POS taggers for Luyia varieties using data from other Luyia languages, exploring both direct transfer and data augmentation strategies.

Methods

The authors employ a Support Vector Machine (SVM) based approach for POS tagging. Their method focuses on a character n-gram model, which is robust for morphologically rich languages where word forms can vary significantly. The features extracted for each word include character unigrams, bigrams, trigrams, and 4-grams internal to the word, as well as the first three characters of the preceding and following words (or the whole word if shorter).

Two main experimental setups are used:

  1. Direct Source to Target Tagging: A model is trained on data from one Luyia variety (the "source") and then directly applied to tag data from another Luyia variety (the "target," specifically Wanga).
  2. Wanga and Source Mixture to Wanga: Data from a source Luyia variety is mixed with a small amount of target Wanga data to train a model. This explores how augmenting a small target language corpus with data from a related language impacts performance.

Datasets

The primary datasets used are corpora for three Luyia varieties:

  • Wanga (lwg): Used as the target language for all experiments.
  • Bukusu (bxk): A more closely related Luyia variety.
  • Tiriki (ida): A more distantly related Luyia variety.

These corpora were extracted from a Fieldworks database and manually annotated with POS tags. A Swahili corpus (Helsinki Corpus of Swahili) was also used as a baseline for comparison, though it differs significantly in annotation methods and composition.

Key Insights and Ideas

  • Linguistic Proximity Isn't Everything: The most surprising finding is that Tiriki, a more distantly related Luyia variety, consistently outperformed Bukusu (a closer relative) in tagging Wanga, both in direct transfer and mixture scenarios. This challenges the intuition that closer languages will always yield better transfer results.
  • Vocabulary Overlap Matters: The authors hypothesize that a higher percentage of exact word matches (Levenshtein distance of 0) between the Tiriki and Wanga corpora, compared to Bukusu and Wanga, contributes to Tiriki's superior performance.
  • Bias Towards Open Classes: The Tiriki classifier appears to have a beneficial bias towards predicting open-class words (nouns, verbs), which are more frequent and informative for tagging. This contrasts with the Bukusu tagger, which sometimes misclassifies words as closed-class items (pronouns, conjunctions).
  • Augmentation Benefits Small Datasets: Mixing source language data with a small target language corpus significantly improves tagging accuracy, especially at smaller training set sizes. However, this benefit diminishes or becomes negligible as the target language data increases.
  • Borrowed Words as a Factor: Loanwords from English or Swahili, which are more prevalent in the Tiriki corpus, might have contributed to its better performance. The character n-gram model, by relying on token patterns, might have implicitly learned better from these borrowed words if they were present in Tiriki's training data.

Key Results

  • Direct Tagging: Tiriki achieved higher accuracy than Bukusu across all training set sizes when directly tagging Wanga. For the largest training set size (0.5 proportion), Tiriki reached an accuracy of 63.07%, while Bukusu achieved 62.49%. Wanga itself served as the upper bound, reaching 91.06%.
  • Language Mixture Tagging: Augmenting Wanga data with Tiriki data consistently yielded higher accuracies than using Wanga data alone, especially at smaller training set sizes. For a 0.05 training set size, the mixture model with Tiriki achieved 78.40% accuracy, compared to 76.36% for Wanga alone. The Bukusu mixture model showed improvement only at the smallest training set size.
  • Out-of-Vocabulary (OOV) Performance: Tiriki also showed better performance on OOV terms compared to Bukusu, with higher accuracies in both direct and mixture tagging scenarios.

Limitations and Future Work

  • Limitations:
    • The study focuses on only three Luyia varieties, and the findings might not generalize to all 20+ Luyia languages.
  • The authors acknowledge that a more thorough investigation of errors would benefit from judgments from a native Wanga speaker.
  • The exact reasons for Tiriki's outperformance are complex and require further investigation.
  • Future Work:
    • Implement the research for other pairings of the available Luyia languages (e.g., Bukusu to tag Tiriki, Wanga to tag Tiriki).
  • Investigate which specific parts of speech benefit most from mixed language models.
  • Explore combining finite-state methods for open-class words with machine learning approaches for other word classes.
  • Develop annotated corpora for the many Luyia languages that currently lack them, potentially using the findings from this study to guide annotation strategies.

Summary updated

BibTeX
@inproceedings{steimel2018part,
  author = {Kenneth Steimel},
  title = {{Part of Speech Tagging in Luyia: A Bantu Macrolanguage}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}

Loading related content...

Figure