A Likelihood Ratio Test of Genetic Relationship among Languages

V. S. D. S. Mahesh Akavarapu, Arnab BhattacharyaNAACL 2024 · DOI

TL;DR

This paper introduces a novel Likelihood Ratio Test (LRT) for determining genetic relationships between languages, inspired by phylogenetic methods in biology. Unlike previous permutation-based tests, this LRT effectively avoids false positives when analyzing groups of languages, providing stronger evidence for proposed macro-families.

What Problem Does This Paper Address?

Identifying genetic relationships between languages is crucial for understanding human history and migration. Lexical similarities (cognates) are a primary indicator, but these can arise by chance. Existing statistical tests, particularly those based on permutation of wordlists, often struggle with analyzing groups of languages or entire language families. They can be computationally infeasible, prone to false positives (claiming relationships where none exist), or false negatives (missing genuine relationships). This paper aims to address these limitations by proposing a more robust and reliable method for assessing linguistic relatedness across multiple languages simultaneously.

Methods

The core of this paper is the development of a Likelihood Ratio Test (LRT). This method draws inspiration from molecular phylogenetics. The key idea is to model language evolution as a process on a phylogenetic tree. The test compares two hypotheses:

  • Null Hypothesis (H₀): A group of languages has a low proportion of "invariant sites" (positions in an aligned wordlist that remain constant or change very little across languages). This suggests a lack of close genetic relationship.
  • Alternative Hypothesis (H<0xE2><0x82><0x90>): A group of languages has a high proportion of invariant sites. This suggests a common ancestor and thus a genetic relationship.

The test works by constructing maximum likelihood phylogenetic trees under both hypotheses and comparing their likelihoods. The ratio of these likelihoods, transformed into a statistic (δ), is used to determine if the evidence supports the alternative hypothesis (i.e., genetic relatedness). A crucial aspect is that this method does not require the reconstruction of proto-languages, which is often a contentious and difficult step in traditional historical linguistics.

Datasets

The study utilizes several datasets:

  • Existing language family data: Datasets for various language families (Afrasian, Dravidian, Indo-European, Kartvelian, Lolo-Burmese, Mayan, Mixe-Zoque, Mon-Khmer, Munda, Uto-Aztecan, Austro-Asiatic, Austronesian, Pama-Nyungan, Sino-Tibetan) were used. Some of these were extracted from existing sources (e.g., Rama et al., 2018), while others were compiled by the authors.
  • Nostratic and Macro-Mayan data: Specific groupings were created to test proposed macro-families, including Dravidian-Indo-European, Dravidian-Indo-European-Kartvelian, Mayan-Mixe-Zoque, Mayan-Uto-Aztecan, and Mayan-Mixe-Zoque-Uto-Aztecan.
  • Data for tree construction task: Datasets from Rama et al. (2018) were used to evaluate the phylogenetic tree construction capabilities of the proposed method against established benchmarks.

Key Insights and Ideas

  • Invariant sites as a proxy for relatedness: The core insight is that genetically related languages, like biological organisms, are expected to share more conserved features (invariant sites in aligned lexicons) than unrelated ones. This provides a novel, quantitative measure of relatedness.
  • Avoiding proto-language reconstruction: The method bypasses the need for controversial proto-language reconstructions, a significant advantage over many traditional linguistic methods.
  • Robustness against false positives: A major finding is that the proposed LRT effectively eliminates false positives, a common problem with previous permutation-based tests when applied to groups of languages.
  • Sensitivity to anomalies: The LRT is shown to be highly sensitive, capable of detecting when the addition of even a single unrelated language can significantly alter the perceived relationship within a group. This is presented as a desirable feature for robust classification.
  • Probabilistic methods outperform distance-based methods for tree construction: The study suggests that probabilistic approaches (like the proposed LRT) are superior to distance-based methods for building phylogenetic trees from linguistic data.

Key Results

  • Elimination of False Positives: The LRT demonstrated a complete absence of false positives in significance testing across various language families, a stark contrast to permutation-based methods which exhibited them in almost all tested cases.
  • Support for Macro-Families: The LRT provided strong supporting evidence for the existence of macro-families such as Dravidian-Indo-European and Macro-Mayan. It also offered weaker but still notable evidence for Nostratic (specifically, the Dravidian-Indo-European-Kartvelian grouping).
  • Tree Construction Performance: While the proposed method's tree construction was slightly behind state-of-the-art Bayesian methods, it performed competitively, especially when using consonant-class character matrices.
  • Sensitivity of LRT: The mean LRT statistic (δ) was observed to be weak (close to zero or negative) for certain groupings (e.g., Drav-IE-Kart), indicating that the alternative hypothesis (higher invariant sites) was not preferred. This highlights the test's ability to detect weak or questionable relationships.

Limitations and Future Work

  • Parameter values for P<0xE1><0xB5><0xA2><0xE1><0xB5><0x9B><0xE1><0xB5><0x9B> and P<0xE1><0xB5><0x9B><0xE1><0xB5><0x9B><0xE1><0xB5><0x9B>: The paper acknowledges that the specific values chosen for the proportion of invariant sites under the null and alternative hypotheses (P<0xE1><0xB5><0xA2><0xE1><0xB5><0x9B><0xE1><0xB5><0x9B> and P<0xE1><0xB5><0x9B><0xE1><0xB5><0x9B><0xE1><0xB5><0x9B>) were roughly decided based on limited examples. Further research is needed to find optimal values, potentially through more extensive data and calibration with phylogenetic software.
  • Inclusion of Uralic languages: For the Nostratic analysis, Uralic languages were excluded due to the criterion of languages being attested before the 10th century CE. Future work could explore including them by considering languages attested around the same period.
  • Semantic Shifts: The paper notes that semantic shifts (words changing meaning over time) were not explicitly addressed. While this can introduce data manipulation, some semantic slots (like 'bark'-'skin') are consistently found to have common words and could potentially be merged.
  • Calibration for different software: The exact values of the test parameters might require calibration depending on the specific phylogenetic software used.

Summary updated

BibTeX
@inproceedings{akavarapu2024likelihood,
  author = {V. S. D. S. Mahesh Akavarapu and Arnab Bhattacharya},
  title = {{A Likelihood Ratio Test of Genetic Relationship among Languages}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.141},
}
Figure