Abstract: Alexandre Bouchard-Côté, Thomas L. Griffiths, Dan Klein. Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL).
2009.
Introduction
Computational historical linguistics has long sought to automate the "comparative method"—the process by which linguists reconstruct extinct ancestral languages (protolanguages) by comparing cognate words across modern descendants. This paper presents a significant advancement in this field, introducing an unsupervised system that "reverse-engineers" ancient word forms with unprecedented scale and accuracy. By integrating linguistic theory with advanced probabilistic modeling, the authors overcome traditional hurdles related to computational complexity and phonological plausibility.
The Computational Comparative Method
The core challenge in automated reconstruction lies in three primary areas:
- Scale: Traditional stochastic models often become computationally intractable when dealing with more than a few languages simultaneously.
- Phonological Universals: Many models fail to incorporate the fact that certain sound changes (e.g., $p \to b$) are naturally more frequent than others (e.g., $p \to r$).
- Internal Consistency: Previous methods often focused solely on "faithfulness" (how much a child word resembles its parent) while ignoring "markedness" (whether the resulting word follows the phonotactic rules of the target language).
Modeling Language Evolution
The authors treat language evolution as a generative process occurring along a phylogenetic tree. Their model balances two primary forces:
- Markedness and Faithfulness: Drawing from Optimality Theory, the model uses faithfulness features to ensure ancestral resemblance and markedness features to ensure the reconstructed word follows the sound combination rules of that specific language.
- Universal Feature Sharing: Instead of learning independent sound changes for every branch, the model utilizes a global weight vector. This allows it to learn universal linguistic tendencies, which is particularly beneficial when data for specific languages is sparse.
- Ancestry Resampling: To handle large, unbalanced trees, the authors implemented a specialized MCMC technique called Ancestry Resampling. This allows for updating entire "slices" of the tree simultaneously, making large-scale reconstruction feasible.
Datasets and Evaluation
The system was evaluated across three distinct language families:
- Austronesian: Reconstructing Proto-Oceanic (POc) using 64 modern descendant languages—an order of magnitude larger than previous automated studies.
- Proto-Malayo-Javanic (PMJ): A smaller 4-language dataset used for benchmarking against existing systems.
- Romance: Reconstructing Classical Latin from modern descendants like Italian, Spanish, and Portuguese.
Results and Scaling
A key finding of this work is that, unlike previous models where more data often introduced noise, this system's performance consistently improves as more languages are added. On the Proto-Oceanic task, the full system achieved a mean edit distance of 1.87 from gold-standard reconstructions. Ablation studies highlighted that markedness was the most critical innovation; removing it caused the error rate to jump significantly.
Discussion and Future Work
While the model represents a major step forward, it currently does not account for complex phenomena such as reduplication, borrowing (loanwords), or chained sound shifts. Additionally, the model assumes a fixed, known phylogenetic tree. Future work aims to simultaneously induce both the tree structure and the ancestral word forms, further automating the process of linguistic discovery.
Summary updated