Toward a deep dialectological representation of Indo-Aryan

Chundra CathcartNAACL 2019

TL;DR

This paper introduces a novel "deep" generative model for understanding the complex relationships within Indo-Aryan languages, moving beyond traditional phylogenetic and dialectometric approaches. By incorporating deep learning architectures, the model effectively disentangles historical language contact from language-specific sound changes, offering a more nuanced view of dialectal evolution.

What Problem Does This Paper Address?

Understanding the historical relationships between languages is a fundamental challenge in linguistics. While phylogenetic methods excel at tracing deep evolutionary paths for distantly related languages, and dialectometry focuses on fine-grained differences between closely related dialects, there's a gap for language groups that have experienced significant contact and admixture. Indo-Aryan languages, despite a common ancestor (Sanskrit), exhibit complex interrelationships due to prolonged contact, making it difficult to disentangle shared history from borrowing. Traditional methods struggle to capture these nuances, leading to an incomplete picture of their dialectal landscape. This paper aims to bridge this gap by developing a model that can explicitly account for both historical divergence and pervasive language contact.

Methods

The core of this work is a novel Bayesian mixture model that extends traditional approaches. The authors propose two main model architectures:

  • Shallow Model: This serves as a baseline, treating sound changes as flat, independent features. It's akin to a Naïve Bayes classifier with a language-specific prior.
  • Deep Model: This is the paper's main contribution. It draws inspiration from deep generative models and neural networks. Crucially, it introduces a hidden layer between the ancestral Indo-Aryan (OIA) etymon and its modern Indo-Aryan (NIA) reflex. This hidden layer, implemented using a Recurrent Neural Network (RNN), acts as an intermediate representation, allowing the model to capture more complex, context-dependent sound change patterns and mimic a "noisy" reconstruction of intermediate linguistic stages. This deep architecture aims to disentangle language-specific sound changes from those influenced by dialect group-level trends and historical contact.

Datasets

The primary dataset used is derived from Turner's (1962–1966) Comparative Dictionary of the Indo-Aryan Languages (CDIAL). This includes OIA headwords and their corresponding NIA reflexes. The authors processed this data by normalizing transcriptions to IPA, accounting for morphological mismatches, and aligning OIA-NIA word pairs using the Needleman-Wunsch algorithm. They excluded cognate sets with fewer than 10 forms, resulting in 33,231 modern Indo-Aryan forms. Geographic metadata from Glottolog was also incorporated.

Key Insights and Ideas

  • The "Deep" Advantage: The deep model, with its intermediate hidden layer, significantly outperforms the shallow model in capturing nuanced dialectal relationships. This suggests that modeling intermediate stages of sound change is crucial for understanding languages with extensive contact.
  • Component Collapse in Shallow Models: The shallow model suffers from "component collapse," where each language is assigned to a single dialect group with very high probability. This indicates that it fails to differentiate between languages effectively, likely due to the high entropy of its learned sound change distributions.
  • Geographic Signal: The deep model and related deep learning architectures (LSTM Autoencoder, LSTM Encoder-Decoder) show a stronger correlation with geographic distance than with genetic (phylogenetic) distance. This suggests that while these models capture diachronic sound changes, the signal they pick up on is more influenced by spatial proximity and contact than by deep evolutionary splits.
  • Disentangling Contact and Evolution: The core idea is that the deep architecture allows for a more sophisticated representation of sound change, enabling the model to better distinguish between changes that are part of a language's independent evolution versus those driven by contact with other dialects or languages.

Key Results

  • The shallow model exhibits "component collapse," assigning languages to single dialect groups with > .99 probability, rendering it largely uninformative for distinguishing dialectal relationships.
  • The deep model, in contrast, infers highly divergent language-level posterior distributions over cluster membership, indicating a better ability to differentiate between languages.
  • While correlations with genetic distance are generally poor across models, the deep model and LSTM-based approaches show significant correlations with geographic distance. For instance, the deep model's Jensen-Shannon divergence shows a strong association with geographic distance (Spearman's ρ = 0.147*).
  • The average entropy of the deep model's dialect-level distributions over hidden-layer activations is 0.86, which is lower than the shallow model's 0.91, suggesting sparser and potentially more meaningful distributions.

Limitations and Future Work

  • Computational Cost: Enumerating all possible reconstructions for the deep model is computationally infeasible, leading to the use of a simplified RNN architecture. The authors acknowledge that more complex architectures might be beneficial but are currently too costly.
  • Discrete vs. Continuous Latent Variables: The current models use discrete latent variables, reflecting traditional linguistic notions of borrowing between discrete dialect groups. Future work could explore continuous-space models for greater flexibility.
  • Prior Distributions and Hyperparameters: The authors suggest that modifying prior distributions and tweaking hyperparameters could lead to more informative results.
  • Bidirectional Architectures and Higher Dimensions: Incorporating bidirectional RNNs and increasing the dimensionality of the hidden layer (J) are identified as promising avenues for future research to capture more complex dependencies.
  • Generalization: The authors hope that this approach, with refinement, can generalize to understanding the history of the Indo-Aryan speech community and be applied to other linguistic scenarios.

Summary updated

BibTeX
@inproceedings{cathcart2019toward,
  author = {Chundra Cathcart},
  title = {{Toward a deep dialectological representation of Indo-Aryan}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
}
Figure