Evaluating Neural Morphological Taggers for Sanskrit

Ashim Gupta, Amrith Krishna, Pawan Goyal, Oliver Hellwig

Abstract

Sanskrit's morphological complexity leads to a vast label space, posing a significant challenge for traditional tagging models. This study evaluates various neural architectures, demonstrating that treating tags as composite features rather than monolithic units significantly improves performance. A hierarchical multi-task learning (MTL) approach achieves state-of-the-art results, though linguistic syncretism remains a persistent challenge.

Introduction

In natural language processing, morphological tagging involves identifying grammatical properties such as gender, case, and number. For a "fusional" language like Sanskrit, this task is exceptionally difficult. Unlike English, which relies heavily on word order, Sanskrit uses complex suffixes to signal syntactic roles. This leads to a massive label space; the Digital Corpus of Sanskrit (DCS) contains over 40,000 possible tag combinations.

The primary challenge is data sparsity. Monolithic taggers, which treat each unique combination as a single atomic label, cannot predict tags they haven't seen during training. This study investigates whether neural architectures that leverage the internal compositionality of tags can better generalize to rare or unseen grammatical forms.

Methodology

The researchers evaluated five neural architectures, all utilizing a hybrid word-and-character-level encoder to capture both global context and sub-word patterns (e.g., suffixes). Additionally, the inclusion of the "last character of the stem" as a feature helped models identify declension types, effectively incorporating traditional linguistic knowledge.

  1. Monolithic (MonSeq): A baseline Bi-LSTM with a CRF layer that treats tags as indivisible units.
  2. Neural Factorial CRF (FCRF): Models dependencies between different morphological features.
  3. Sequence Generation (Seq): Uses an LSTM decoder to generate features one by one.
  4. Multi-Task Learning (MTL-Shared): Predicts each feature as a separate task using a shared encoder.
  5. MTL-Hierarchy: The most advanced approach, establishing a hierarchy where fundamental features (like Number) are predicted at lower layers to provide context for complex features (like Tense) at higher layers.
Neural architectures for Sanskrit tagging.

Experimental Setup

The models were trained and tested on the Digital Corpus of Sanskrit (DCS), using 50,000 sentences for training and 11,000 for testing. A specialized "Unseen Test Set" was also used, containing sentences with at least one tag that never appeared in the training data. The tagset comprised 71 features across categories like Nouns, Verbs, and Participles. Notably, the training data only covered 2,757 unique labels out of a theoretical 42,606, highlighting the extreme sparsity.

Results and Analysis

The results demonstrate that compositionality is essential for Sanskrit tagging. The monolithic model (MonSeq) performed poorly, especially on rare tags. In contrast, composite models could generalize; for example, predicting a "Nominative Plural" by combining knowledge of "Nominative" and "Plural" seen in other contexts.

Comparative performance of tagging systems.

The MTL-Hierarchy model emerged as the winner, achieving 86.74% Token Accuracy and a 94.11 Macro F1-score. This confirms that predicting grammatical features in a specific order helps the model resolve complex dependencies. On the "unseen" tag set, composite models achieved an F1-score of approximately 55%, a significant feat compared to the 0% capability of the monolithic baseline.

The Challenge of Syncretism

Despite these advances, syncretism remains a major hurdle. Syncretism occurs when different grammatical functions share the same surface form (e.g., a word that could be either Nominative or Accusative). All models saw performance drops to the 68-70% range for syncretic tokens, indicating that current architectures still struggle to fully utilize sentence context to resolve these ambiguities.

Conclusion

Treating Sanskrit morphological tags as composite features significantly improves tagging accuracy and generalization. While hierarchical multi-task learning provides the best performance, future work must focus on better handling of syncretism, perhaps by integrating external linguistic resources like lexicons or rule-based parsers to supplement neural learning.

Summary updated

Abstract

Neural sequence labelling approaches have achieved state of the art results in morphological tagging. We evaluate the efficacy of four standard sequence labelling models on Sanskrit, a morphologically rich, fusional Indian language. As its label space can theoretically contain more than 40,000 labels, systems that explicitly model the internal structure of a label are more suited for the task, because of their ability to generalise to labels not seen during training. We find that although some neural models perform better than others, one of the common causes for error for all of these models is mispredictions due to syncretism.
BibTeX
@inproceedings{gupta2020evaluating,
  author = {Ashim Gupta and Amrith Krishna and Pawan Goyal and Oliver Hellwig},
  title = {{Evaluating Neural Morphological Taggers for Sanskrit}},
  booktitle = {SIGMORPHON},
  year = {2020},
  doi = {10.18653/v1/2020.sigmorphon-1.23},
}
Figure