Letters From the Past: Modeling Historical Sound Change Through Diachronic Character Embeddings

Sidsel Boldsen, Patrizia PaggioACL 2022 · DOI

Historical linguistics has traditionally relied on the manual analysis of ancient manuscripts to track sound change—a labor-intensive process often hampered by data sparsity. In a paper presented at ACL 2022, researchers proposed a novel computational framework to automate this process. By adapting the "distributional hypothesis" to the character level, they successfully modeled centuries of phonological evolution in Danish using diachronic character embeddings.

The Distributional Hypothesis for Phonology

The core premise of the study is that characters, much like words, are "known by the company they keep." Just as the meaning of a word is defined by its semantic context, the sound represented by a character is defined by its phonotactic environment. The authors hypothesized that if a sound $A$ evolves into sound $B$ (a process known as sound change), the distributional profile of $B$ in later periods will begin to converge with the historical profile of $A$.

Methodology: PPMI and Diachronic Embeddings

To test this, the researchers utilized Positive Pointwise Mutual Information (PPMI) to create count-based embeddings for characters. Unlike neural embeddings, PPMI is deterministic and naturally aligned across time periods, allowing for direct comparisons of character "neighborhoods" across centuries.

To distinguish genuine historical shifts from random spelling noise, the team employed a Linear Regression model. They compared the target historical corpus against a "control" corpus generated by shuffling the dates of the original texts. A statistically significant interaction between the time bin and the corpus type (real vs. control) serves as evidence of a systematic sound change.

Validation on Simulated Data

Before applying the model to real-world data, the authors validated it using synthetic datasets. This included a modern Danish corpus (UD Treebank) where sound changes were manually injected. The model successfully identified these simulated changes with high statistical significance ($p < 0.001$), demonstrating that character-level embeddings can capture phonological shifts even amidst the noise of real-world vocabulary.

Figure 2: Interaction of Bin and Corpus on Distance in UD Danish. The decreasing distance in the 'Change' corpus indicates successful detection of simulated sound shifts.

Tracking Real-World Sound Change in Danish Place Names

The primary application of the model was the Danmarks Stednavne dataset, containing ~900,000 historical attestations of Danish geographical names from the 12th to the 18th century. The study focused on "lenition" (the softening of consonants), specifically the shifts of $k \to g$, $t \to d$, and $p \to b$.

The model successfully tracked the $k \to g$ and $t \to d$ shifts. For instance, the $k \to g$ shift was highly visible in suffixes like -vig ('inlet'), which appeared as -k in earlier records. However, the model failed to find a significant shift for $p \to b$. The authors attribute this to the rarity of the letter 'p' in Germanic languages, which created data sparsity that hindered the distributional model.

Figure 5: Distance over time for the $t \to d$ shift in historical Danish. The divergence between the real 'Change' corpus and the shuffled 'Control' corpus highlights the systematic nature of the sound change.

Discussion and Future Directions

While successful, the authors noted that the model is sensitive to "cultural drift"—changes in the popularity of specific names that might mimic sound change. Furthermore, the current count-based approach treats every character string as a unique dimension, failing to recognize relationships between similar characters (e.g., 'i' and 'ii').

Future work may involve dense neural embeddings, which could better generalize across phonological contexts. Nonetheless, this research proves that "spelling as a proxy for sound" is a viable path for computational historical linguistics, offering a scalable way to peer into the phonetic past through the lens of historical text.

Summary updated

BibTeX
@inproceedings{boldsen2022letters,
  author = {Sidsel Boldsen and Patrizia Paggio},
  title = {{Letters From the Past: Modeling Historical Sound Change Through Diachronic Character Embeddings}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.acl-long.463},
}
Figure