Inferring Morphological Complexity from Syntactic Dependency Networks: A Test

Guglielmo Inglese, Luca Brigada VillaNAACL 2021

TL;DR

This paper investigates whether the structure of syntactic dependency networks can serve as a proxy for a language's morphological complexity, specifically its analyticity/syntheticity. The authors find that while some network features like the number of nodes and average path length show promise, their reliability is significantly affected by treebank size, suggesting that current methods may not yet yield consistently interpretable results for large-scale cross-linguistic analysis.

What Problem Does This Paper Address?

Linguistic typology has long sought to classify languages based on their morphological complexity. Historically, this led to discrete categories like "isolating" (analytic) and "agglutinative" (synthetic). However, modern linguistics views these as poles of a continuum, with languages exhibiting varying degrees of both strategies. Quantitatively measuring this morphological complexity across many languages is challenging. Traditional methods rely on manual annotation of linguistic features, which is labor-intensive and limits scalability. This paper explores an alternative: using the structure of syntactic dependency networks, derived from automatically parsed treebanks, as an indirect measure of morphological complexity. The core problem is to determine if network properties can reliably reflect a language's tendency towards analytic (using free markers) or synthetic (using bound markers) strategies, and to what extent this approach is scalable and robust.

Methods

The authors build upon previous work by Liu & Xu (2011) and propose using syntactic dependency networks to infer morphological complexity. A syntactic dependency network represents words as nodes and grammatical dependencies between them as edges. The study focuses on two specific network properties: the number of nodes and the average path length.

The methodology involves:

  1. Data Sampling: A sample of 42 languages was selected from the Universal Dependencies (UD) project, chosen for its consistent annotation schema and availability of treebanks.
  2. Control Group: A subset of languages was designated as a "control group" to represent the extremes of analyticity (e.g., Vietnamese, Mandarin Chinese) and syntheticity (e.g., Russian, Finnish).
  3. Treebank Subsampling: To address the issue of varying treebank sizes, the authors created random subsamples of tokens (1k, 5k, 10k, 20k, 30k, 50k, 75k) from the control group treebanks.
  4. Network Induction: Word-based dependency networks were induced from these subsamples, excluding punctuation and elliptical dependencies.
  5. Statistical Testing: A Welch t-test was used to compare the network indexes (number of nodes, average path length, etc.) between the analytic and synthetic groups across different subsample sizes. The goal was to identify indexes that consistently differentiate the two groups.
  6. Analysis of Sample Languages: The selected indexes were then calculated for the remaining languages in the sample and visualized to observe their position along the analyticity/syntheticity continuum.

Datasets

The study utilizes treebanks from the Universal Dependencies (UD) project. The authors selected 42 languages for which UD treebanks are available. Specific treebanks are listed in Appendix A, along with their token sizes. The sampling procedure aimed for diversity across language families and included historical varieties where possible. No new benchmarks were introduced; rather, existing UD treebanks were used for analysis.

Key Insights and Ideas

  • Network Structure as a Proxy: The fundamental idea is that languages with more synthetic morphology might require more explicit grammatical marking, leading to more dependencies and thus more nodes and potentially longer paths in their syntactic dependency networks. Conversely, analytic languages, relying on free words, might exhibit simpler network structures.
  • Analyticity/Syntheticity as a Continuum: The paper reinforces the view that morphological complexity is not a binary classification but a gradient, and network analysis can potentially map languages onto this continuum.
  • The Impact of Treebank Size: A crucial finding is that the reliability of network indexes as proxies for morphological complexity is highly sensitive to the size of the treebank used. Smaller subsamples (up to 30k tokens) showed more consistent results, while larger samples or the full treebanks yielded less interpretable patterns. This suggests that current methods might struggle with the inherent variability and noise in larger, more diverse corpora.
  • Average Path Length and Number of Nodes as Promising Indicators: Among the tested network indexes, the number of nodes and average path length showed the most consistent ability to differentiate between the control groups of analytic and synthetic languages, particularly at smaller treebank sizes.

Key Results

  • The number of nodes and average path length were found to be the most reliable network indexes for distinguishing between analytic and synthetic languages, especially when using treebanks of up to 30,000 tokens.
  • For the control group, languages classified as synthetic generally showed a higher number of nodes and a longer average path length compared to analytic languages, aligning with the authors' hypothesis.
  • When analyzing the full sample of 42 languages, especially with larger treebanks (30k tokens and above), the patterns became less clear. Some highly inflectional languages (e.g., Latin, Ancient Greek) appeared to cluster with analytic languages, contradicting expectations.
  • The study found that network indexes did not yield consistently interpretable results for morphological complexity when applied to larger cross-linguistic datasets, suggesting limitations in the current approach.

Limitations and Future Work

The authors acknowledge several limitations:

  • Treebank Size Sensitivity: The most significant limitation is the observed sensitivity of network indexes to treebank size. This makes it difficult to draw robust conclusions for languages with very large treebanks or to compare results across different sizes reliably.
  • Limited Scope of Network Indexes: The methodology can only infer certain aspects of morphological complexity (analyticity/syntheticity) and cannot capture finer-grained distinctions like cumulative vs. agglutinative strategies.
  • Potential Impact of Annotation Schemas: While UD provides a consistent schema, variations in how specific grammatical features are annotated could still influence network structures.
  • Working with Full Networks: The study extracted networks from entire treebanks. Future work could explore networks derived from specific parts of speech or finer-grained linguistic categories.

Future work suggested includes:

  • Further research to understand the interplay between treebank size and network indexes, possibly using different statistical tests.
  • Investigating the impact of annotation schemas and exploring networks derived from more granular linguistic units.
  • Testing whether better results can be achieved by analyzing networks based on specific parts of speech.
  • Developing more robust methods to overcome the size sensitivity issue and ensure consistent interpretability across diverse datasets.

Summary updated

BibTeX
@inproceedings{inglese2021inferring,
  author = {Guglielmo Inglese and Luca Brigada Villa},
  title = {{Inferring Morphological Complexity from Syntactic Dependency Networks: A Test}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure