TL;DR
This paper introduces the Vedic Compound Dataset (VCD), the first resource of its kind for Vedic Sanskrit, a crucial ancient language. The VCD aims to facilitate the study of language change and offers comparative material for cross-linguistic research on compounds, a complex linguistic phenomenon.
What Problem Does This Paper Address?
Understanding how words combine to form new meanings (compounding) is fundamental to linguistics. While classical and Neo-Sanskrit have been studied, Vedic Sanskrit, the earliest form of the language (ca. 1500-500 BCE), has lacked a dedicated, annotated dataset for compound analysis. This gap hinders the study of language evolution, particularly the development of Indo-European languages. Vedic compounds are particularly challenging to analyze automatically because they exhibit complex internal structures and their grammatical marking isn't always consistent, making it difficult to distinguish between different types of compounds, especially endocentric (where the meaning is contained within the compound) and exocentric (where the meaning is external).
Methods
The authors collected and annotated compounds from the Vedic Treebank (VTB) and the Digital Corpus of Sanskrit (DCS). They extended existing Universal Dependencies (UD) annotation guidelines to explicitly capture the internal syntactic structure of compounds. For classification, they developed a partly rule-based approach combined with human expertise. This involved using internal UD labels to distinguish coordinate compounds and then focusing on the endocentric/exocentric distinction for subordinate and attributive compounds. This distinction was achieved through a combination of automatic methods (e.g., gender mismatch) and manual annotation for ambiguous cases. They also experimented with a simple multinomial regression model to test if syntactic information could predict compound types.
Datasets
- Vedic Compound Dataset (VCD): A newly introduced dataset containing almost 7,000 two-word compounds from Vedic Sanskrit. It includes information on morphology, internal and external syntactic relations, chronology, and Vedic subtraditions.
- Digital Corpus of Sanskrit (DCS): Used for lexical and morphosyntactic annotations.
- Vedic Treebank (VTB): Provided UD annotations for approximately 32,000 sentences, which were supplemented with enhanced compound annotation guidelines.
Key Insights and Ideas
- Vedic Compounds are Rich and Complex: The VCD reveals that Vedic compounds exhibit a wide range of syntactic structures and a significant proportion (41.8%) are exocentric, which is higher than in many modern languages.
- Diachronic Trends are Evident: The dataset shows a clear diachronic shift in the endo-/exocentric dimension of compounds, with a higher rate of exocentric compounds in the archaic Rig Vedic period compared to later periods.
- Internal Syntactic Relations are Crucial: The authors found that compound-internal syntactic labels are the most important predictors for automatic compound classification, more so than external sentence-level relations.
- UD Limitations for Compounds: Standard UD guidelines are insufficient for detailed compound analysis, highlighting the need for extended annotation schemes.
Key Results
- The VCD contains nearly 7,000 two-word compounds.
- Exocentric compounds constitute 41.8% of the VCD, a notably high percentage compared to other languages.
- A simple multinomial regression model achieved F-scores above 80% for compound classification, demonstrating the effectiveness of syntactic information.
- Ablation tests showed that removing compound-internal syntactic labels significantly degraded the model's performance.
Limitations and Future Work
- Incomplete Coverage: The VCD is based on approximately 35% of extant Vedic literature.
- Lexicalized Compounds Excluded: Compounds treated as lexicalized in the DCS were not included, limiting the dataset's scope.
- Two-Word Compounds Only: The current version is restricted to two-word compounds, with plans to include longer ones in the future.
- POS Tag Reliability: POS tags from the VTB are not entirely reliable, though manual corrections were made.
Future work includes expanding the dataset to include longer compounds and lexicalized forms, and exploring the potential for automatic recognition of exocentricity through improved UD labels and deep learning models.
Summary updated