Evaluating Neural Morphological Taggers for Sanskrit

Ashim Gupta, Amrith Krishna, Pawan Goyal, Oliver Hellwig

Abstract

Neural sequence labelling approaches have achieved state of the art results in morphological tagging. We evaluate the efficacy of four standard sequence labelling models on Sanskrit, a morphologically rich, fusional Indian language. As its label space can theoretically contain more than 40,000 labels, systems that explicitly model the internal structure of a label are more suited for the task, because of their ability to generalise to labels not seen during training. We find that although some neural models perform better than others, one of the common causes for error for all of these models is mispredictions due to syncretism.

Introduction

Sanskrit is a morphologically rich language where a single word can convey extensive grammatical information through complex affixation. In traditional sequence labeling, this leads to a massive label space—theoretically exceeding 40,000 unique tags. Standard "monolithic" taggers treat each unique combination of features (e.g., Noun-Masculine-Singular-Nominative) as a single atomic class. This approach suffers from extreme data sparsity, as many valid grammatical combinations may never appear in training data. This paper investigates whether "composite" models, which treat tags as collections of individual features like gender, case, and number, can better generalize to unseen forms.

Methodology

The researchers compared a monolithic baseline against four composite architectures. All models utilize a shared Bi-LSTM encoder with word and character-level embeddings to capture contextual and sub-word information.

  1. MonSeq (Monolithic): A standard Bi-LSTM-CRF baseline that treats each full tag as a single class.
  2. Seq (Sequence Generation): Employs an LSTM decoder to generate morphological features as a sequence for each token.
  3. FCRF (Factorial CRF): Models dependencies between different features (e.g., the relationship between Gender and Case) using a factorial Conditional Random Field.
  4. MTL-Shared (Multi-Task Learning): Treats each feature as an independent task with a shared encoder but separate output layers.
  5. MTL-Hierarchy: A sophisticated multi-task approach where features are predicted at different levels of the network. "Low-level" features are predicted at shallower layers, while "high-level" features are predicted at deeper layers, using shortcut connections to propagate information.
Architectural diagrams for (a) FCRF, (b) Seq, and (c) MTL-Hierarchy models.

Experimental Setup

The study utilized the Digital Corpus of Sanskrit (DCS), consisting of a training set of 50,000 sentences and a test set of 11,000 sentences. A specialized "unseen" test set was also used, containing words with morphological tags never encountered during training. The tagset included features such as Tense (T), Case (C), Number (N), Gender (G), Person, and the "Last Character" (L) of the stem.

Key Results and Discussion

Composite models significantly outperformed the monolithic baseline across all metrics. The MTL-Hierarchy model emerged as the top performer, suggesting that modeling the natural dependency between morphological features (such as Number influencing Tense) provides a significant advantage.

Performance comparison of different tagging systems across various metrics.

Generalization and Syncretism

A critical finding was the ability of composite models to generalize. On the unseen tags test set, composite models maintained a Macro F1 of approximately 55, whereas the monolithic MonSeq model scored 0 because it cannot predict labels absent from the training set.

However, all models encountered a performance ceiling due to syncretism—instances where different grammatical forms of a word are phonologically or orthographically identical. In Sanskrit nouns, syncretism accounts for 20% to 25% of all mispredictions. Additionally, models showed a surface-level bias, often over-relying on the last character of a word rather than deeper contextual cues.

Conclusion

Modeling the internal structure of morphological tags is essential for highly inflected languages like Sanskrit. While hierarchical multi-task learning provides the best performance and enables generalization to unseen tags, syncretism remains a primary challenge. Future work may require better modeling of long-range syntactic dependencies or the integration of lexicon-driven constraints to further improve accuracy and prevent linguistically impossible feature combinations.

Summary updated

Abstract

Neural sequence labelling approaches have achieved state of the art results in morphological tagging. We evaluate the efficacy of four standard sequence labelling models on Sanskrit, a morphologically rich, fusional Indian language. As its label space can theoretically contain more than 40,000 labels, systems that explicitly model the internal structure of a label are more suited for the task, because of their ability to generalise to labels not seen during training. We find that although some neural models perform better than others, one of the common causes for error for all of these models is mispredictions due to syncretism.
BibTeX
@inproceedings{gupta2020evaluating,
  author = {Ashim Gupta and Amrith Krishna and Pawan Goyal and Oliver Hellwig},
  title = {{Evaluating Neural Morphological Taggers for Sanskrit}},
  booktitle = {SIGMORPHON},
  year = {2020},
  doi = {10.18653/v1/2020.sigmorphon-1.23},
}
Figure