Abstract
Alexandre Bouchard, Percy Liang, Thomas Griffiths, Dan Klein. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL).
2007.
Introduction
Historical linguistics seeks to understand how languages evolve over time, specifically through diachronic phonology—the study of sound change. Traditionally, linguists have relied on the "comparative method," manually comparing cognates (words with a common origin) to reconstruct ancestral proto-languages. While computational methods have previously addressed phylogenetic tree building, they often treated words as binary features, ignoring the rich structural information in phoneme sequences.
This paper presents a generative probabilistic model that simulates sound changes as stochastic edits along a phylogenetic tree. The system automates the reconstruction of lost ancestral words, identifies historical "sound laws," and infers the most likely evolutionary relationships between languages.
Methods
The proposed generative model treats a root word as the starting point for a series of stochastic edits as it descends the branches of a phylogenetic tree.
The Edit Model
Unlike simple edit distance, this model employs context-dependent edits (insertions, deletions, and substitutions). The probability of a change is conditioned on "natural classes" (e.g., vowels vs. consonants), capturing the linguistic reality that sound changes are often regular and context-specific.
The Language Model
A bigram phoneme model is applied at the root to ensure reconstructed ancestral forms are phonologically plausible.
Inference via Monte Carlo EM
Learning parameters (sound laws) and hidden variables (ancestral words) is handled via a Monte Carlo Expectation-Maximization (MC-EM) algorithm. The E-step uses Gibbs sampling with a dynamic programming lattice to sample possible word forms, while the M-step updates edit rule probabilities based on these samples.
Experimental Setup
The model was tested on a Romance language corpus aggregated from Wiktionary, a Bible parallel corpus, and Europarl. The dataset included Latin (ancestor), Italian, Spanish, and Portuguese, resulting in 586 cognate tuples across all four languages after converting text to the International Phonetic Alphabet (IPA).
Results and Discussion
The model demonstrated significant improvements in reconstruction accuracy, outperforming standard edit-distance baselines by 9% for Latin and 11% for Spanish. It successfully captured complex shifts, such as the emergence of the /j/ in Spanish diente from Latin dentis.
Phylogeny Selection
A key capability of the model is its ability to evaluate different tree topologies. By assigning log-likelihood scores to various structures, the model correctly identified the universally accepted Romance tree (Topology 2) over "incorrect" alternatives.
Qualitative Insights
The EM algorithm "rediscovered" known linguistic phenomena, such as the deletion of final /m/ and /s/ during the transition from Classical to Vulgar Latin, and the /v/ to /b/ fortition in Spanish. These findings align closely with established historical linguistics.
Limitations and Future Work
Current limitations include the model's inability to account for lexical drift (word replacement) and borrowing (loanwords). It also struggles with "chained shifts," where changes are interdependent. Future research may involve learning optimal phonetic features or using log-linear models to capture more complex overlapping contexts.
Summary updated