Abstract
Most work on part-of-speech (POS) tagging is focused on high-resource languages, or examines low-resource and active learning settings through simulated studies. We evaluate POS tagging techniques on an actual endangered language, Griko. We present a resource that contains 114 narratives in Griko, along with sentence-level translations in Italian, and provides gold annotations for the test set. Based on a previously collected small corpus, we investigate several traditional methods, as well as methods that take advantage of monolingual data or project cross-lingual POS tags. We show that the combination of a semi-supervised method with cross-lingual transfer is more appropriate for this extremely challenging setting, with the best tagger achieving an accuracy of 72.9%. With an applied active learning scheme, which we use to collect sentence-level annotations over the test set, we achieve improvements of more than 21 percentage points.
Introduction
Natural Language Processing (NLP) research predominantly operates within a "high-resource" bubble, prioritizing languages like English or Mandarin where data is abundant. When low-resource settings are explored, they are often simulated by artificially restricting large datasets. This paper addresses the "messy" reality of Griko, a seriously endangered Greek dialect spoken by fewer than 20,000 people in the Salento region of Southern Italy.
Griko presents unique computational challenges that standard models often fail to capture:
- Orthographic Variance: The lack of a standard spelling system leads to significant noise in string matching.
- Morphological Fusion: Multiple parts of speech often fuse into a single token (e.g., "to the" becoming one word), necessitating complex multi-tagging.
- Code-Switching: Speakers frequently oscillate between Griko, Italian, and the local Salentino dialect.
Methodology and Data
The researchers introduced a significant new resource: a parallel corpus of 114 narratives (approx. 10,000 sentences) paired with Italian translations. A gold-standard test set of 10 narratives was hand-annotated by linguists to provide a reliable benchmark.
To evaluate performance, several architectures were compared:
- Baselines: Stanford Log-linear tagger, a basic Conditional Random Field (CRF), and a neural bi-LSTM.
- Enhanced CRF (crf-mod): Augmented with prefix/suffix features to capture Griko's rich morphology.
- Semi-Supervised Learning (G&B): Utilizing the Garrette & Baldridge framework to leverage unannotated monolingual data via label propagation.
- Cross-Lingual Projection: Projecting POS tags from the high-resource Italian translations onto aligned Griko words.
Key Results and Insights
The study revealed that in extreme low-resource scenarios, hybrid approaches are essential. The best initial performance was achieved by combining semi-supervised learning with cross-lingual transfer from Italian.
As shown in the results table, the G&B model with cross-lingual projection (+clp) reached an accuracy of 72.92%. Interestingly, as the training set grew to roughly 800 sentences, the well-tuned crf-mod began to outperform more complex semi-supervised models, suggesting that feature engineering remains highly effective when data is scarce but growing.
The Impact of Active Learning
To bridge the gap to high-accuracy tagging, the authors implemented an active learning scheme. By iteratively selecting narratives for human annotation and retraining the models, they simulated a realistic workflow for field linguists.
The results were dramatic: accuracy improved by over 21 percentage points, eventually reaching 94.17%. This demonstrates that even a small amount of targeted human intervention can yield massive gains in low-resource NLP settings.
Conclusion
This work provides both a valuable resource for the Griko language and a roadmap for applying NLP to other endangered varieties. While challenges like code-switching and non-standard orthography remain, the success of cross-lingual projection and active learning offers a path forward for preserving linguistic diversity through technology.
Paper Citation: Anastasopoulos, A., Lekakou, M., Quer, J., Zimianiti, E., DeBenedetto, J., & Chiang, D. (2018). Part-of-Speech Tagging on an Endangered Language: a Parallel Griko-Italian Resource. arXiv:1806.03757 [cs.CL].
Summary updated