Boosting Neural POS Tagger for Farsi Using Morphological Information

Peyman Passban, Qun Liu, Andy Way

TL;DR

This paper introduces a novel neural network-based Part-of-Speech (POS) tagger for Farsi that significantly improves accuracy and handles out-of-vocabulary (OOV) words better than existing methods. By incorporating shallow morphological analysis, the tagger achieves state-of-the-art performance, demonstrating the crucial role of morphology in low-resource languages.

What Problem Does This Paper Address?

Farsi, like many languages, suffers from a "low-resource" status, meaning there's a scarcity of readily available data and efficient processing tools. This is particularly problematic for tasks like Part-of-Speech (POS) tagging, which is fundamental for many Natural Language Processing (NLP) applications. While existing Farsi POS taggers have achieved decent accuracy (around 96%), they struggle significantly with out-of-vocabulary (OOV) words – words not present in their training data. This limitation hinders their practical usability. Furthermore, Farsi's rich morphology, with its extensive use of prefixes, suffixes, and variations, presents a unique challenge that traditional taggers often fail to fully leverage. This paper aims to address both the accuracy and OOV word issues by developing a neural POS tagger that effectively utilizes Farsi's morphological richness.

Methods

The core of this work is a Multilayer Perceptron (MLP), a type of neural network, adapted for POS tagging. The authors designed a three-module architecture:

  1. Representation Module: This module converts words into numerical vectors. They experimented with Word2Vec and GloVe embeddings, which capture semantic relationships between words.
  2. Hidden Layers: These layers process the input vectors through non-linear transformations using tanh activation functions, allowing the network to learn complex patterns.
  3. Prediction Layer: This layer uses a Softmax function to output the probability distribution over possible POS tags for a given word.

A key innovation is the integration of shallow morphological analysis. They developed an algorithm to extract prefixes, suffixes, and variations from words, treating these as distinct features. This morphological information is then combined with word embeddings as input to the neural network. The network is trained using stochastic gradient descent with mini-batches and L2 regularization to prevent overfitting.

Datasets

The primary dataset used is the Bijankhan corpus, a widely used Farsi corpus containing approximately 2.6 million manually tagged words. The authors note that while this corpus is standard, it has a very fine-grained tagset (550 tags). For their experiments, they used a reduced tagset of 40 tags, as proposed by the data development team. They also mention the UPC corpus (a modified version of Bijankhan with 2.7M words and 32 POS tags) and the Peykareh corpus (10M tagged words, not publicly available) as other Farsi resources.

Key Insights and Ideas

  • Morphology is King for Farsi: The paper strongly demonstrates that leveraging Farsi's rich morphology is crucial for improving POS tagging accuracy, especially for OOV words.
  • Neural Networks Excel at OOV: Neural networks, with their ability to learn distributed representations, are inherently better at handling OOV words compared to traditional methods.
  • Shallow Analysis is Sufficient: A "shallow" morphological analysis, extracting basic affixes and variations, provides significant gains without requiring a full-fledged, complex morphological analyzer (which is not readily available for Farsi).
  • Feature Engineering Matters: The combination of word embeddings and morphological features as input to the neural network is more effective than using either alone.
  • Context Window Size: For Farsi, a smaller context window (considering only the target word and its immediate preceding words) is sufficient and even preferable, unlike languages like English where larger contexts might be beneficial.

Key Results

The proposed neural POS tagger (NPT) achieved an overall accuracy of 97.37% on the Bijankhan corpus. This surpasses previous state-of-the-art taggers, including HunPos (96.9%). Notably, the NPT achieved an unseen word accuracy of 89.01% for 5.4% OOV words, a significant improvement over existing taggers. The paper also shows that incorporating morphological features (prefix, suffix, variation) incrementally boosted accuracy, with the full set of features leading to a 2.73% improvement over using only word embeddings.

Limitations and Future Work

  • Shallow Analysis Precision: The authors acknowledge that their morphological analysis is "shallow" and not perfectly precise. While it yields considerable improvements, a more sophisticated morphological analyzer could potentially lead to even better results.
  • Runtime vs. Accuracy Trade-off: While their NPT is fast and accurate, they mention that an in-house recurrent POS tagger performed slightly better but had a higher training/running time, suggesting a trade-off that might be acceptable for certain applications.
  • Future Work: The authors express interest in applying their NN-based methods to other Farsi NLP tasks, such as semantic role labeling and named-entity recognition, and exploring sequence-level computations like parsing and translation. They also suggest investigating more complex neural architectures.

Summary updated

Abstract

Farsi (Persian) is a low-resource language that suffers from the data sparsity problem and a lack of efficient processing tools. Due to their broad application in natural language processing tasks, part-of-speech (POS) taggers are one of those important tools that should be considered in this respect. Despite recent work on Farsi tagging, there is still room for improvement. The best reported accuracy so far is 96%, which in special cases can rise to 96.9%. The main problem with existing taggers is their inefficiency in coping with out-of-vocabulary (OOV) words. Addressing both problems of accuracy and OOV words, we developed a neural network-based POS tagger (NPT) that performs efficiently on Farsi. Despite using less data, NPT provides better results in comparison to state-of-the-art systems. Our proposed tagger performs with an accuracy of 97.4%, with performance highly influenced by morphological features. We carry out a shallow morphological analysis and show considerable improvement over the baseline configuration.
BibTeX
@article{passban2016boosting,
  author = {Peyman Passban and Qun Liu and Andy Way},
  title = {{Boosting Neural POS Tagger for Farsi Using Morphological Information}},
  journal = {ACM Trans. Asian Low Resour. Lang. Inf. Process.},
  year = {2016},
  doi = {10.1145/2934676},
}
Figure