A Probabilistic Approach to Diachronic Phonology

Alexandre Bouchard-Côté, Percy Liang, Thomas L. Griffiths, Dan KleinEMNLP 2007 · OpenReview

Abstract

Alexandre Bouchard, Percy Liang, Thomas Griffiths, Dan Klein. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL).

2007.

Introduction

Historical linguistics seeks to understand how languages evolve over time, specifically through diachronic phonology—the study of sound change. Traditionally, linguists have relied on the "comparative method," manually comparing cognates (words with a common origin) to reconstruct ancestral proto-languages. While computational methods have previously addressed phylogenetic tree building, they often treated words as binary features, ignoring the rich structural information in phoneme sequences.

This paper presents a generative probabilistic model that simulates sound changes as stochastic edits along a phylogenetic tree. The system automates the reconstruction of lost ancestral words, identifies historical "sound laws," and infers the most likely evolutionary relationships between languages.

Methods

The proposed generative model treats a root word as the starting point for a series of stochastic edits as it descends the branches of a phylogenetic tree.

The Edit Model

Unlike simple edit distance, this model employs context-dependent edits (insertions, deletions, and substitutions). The probability of a change is conditioned on "natural classes" (e.g., vowels vs. consonants), capturing the linguistic reality that sound changes are often regular and context-specific.

The Language Model

A bigram phoneme model is applied at the root to ensure reconstructed ancestral forms are phonologically plausible.

Inference via Monte Carlo EM

Learning parameters (sound laws) and hidden variables (ancestral words) is handled via a Monte Carlo Expectation-Maximization (MC-EM) algorithm. The E-step uses Gibbs sampling with a dynamic programming lattice to sample possible word forms, while the M-step updates edit rule probabilities based on these samples.

Experimental Setup

The model was tested on a Romance language corpus aggregated from Wiktionary, a Bible parallel corpus, and Europarl. The dataset included Latin (ancestor), Italian, Spanish, and Portuguese, resulting in 586 cognate tuples across all four languages after converting text to the International Phonetic Alphabet (IPA).

Results and Discussion

The model demonstrated significant improvements in reconstruction accuracy, outperforming standard edit-distance baselines by 9% for Latin and 11% for Spanish. It successfully captured complex shifts, such as the emergence of the /j/ in Spanish diente from Latin dentis.

Phylogeny Selection

A key capability of the model is its ability to evaluate different tree topologies. By assigning log-likelihood scores to various structures, the model correctly identified the universally accepted Romance tree (Topology 2) over "incorrect" alternatives.

Comparison of four language tree topologies used to evaluate the model's ability to infer phylogenetic relationships.

Qualitative Insights

The EM algorithm "rediscovered" known linguistic phenomena, such as the deletion of final /m/ and /s/ during the transition from Classical to Vulgar Latin, and the /v/ to /b/ fortition in Spanish. These findings align closely with established historical linguistics.

Limitations and Future Work

Current limitations include the model's inability to account for lexical drift (word replacement) and borrowing (loanwords). It also struggles with "chained shifts," where changes are interdependent. Future research may involve learning optimal phonetic features or using log-linear models to capture more complex overlapping contexts.

Summary updated

BibTeX
@inproceedings{bouchardcote2007probabilistic,
  author = {Alexandre Bouchard-Côté and Percy Liang and Thomas L. Griffiths and Dan Klein},
  title = {{A Probabilistic Approach to Diachronic Phonology}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2007},
  url = {https://openreview.net/forum?id=Bk4evfzuZH},
}
Figure