TL;DR
This paper introduces a novel method for tracking linguistic change in historical texts by analyzing the errors of a neural lemmatization model. By applying this model to Old Literary Finnish texts from different periods, the authors demonstrate how error patterns can reveal evolving language features and suggest a path towards automated historical periodization.
What Problem Does This Paper Address?
Understanding how languages evolve over time is crucial for historical and linguistic studies. Specifically, pinpointing distinct periods of change within a language's history can be challenging. Old Literary Finnish, a written form of Finnish used from the 16th to the early 19th century, presents a rich case study for this. While historical linguistics has methods for analyzing such changes, automating this process and precisely dating linguistic innovations remains an open challenge. This paper aims to bridge this gap by leveraging modern NLP techniques to automatically detect and analyze linguistic shifts in historical texts, offering a more systematic approach to periodization.
Methods
The core of this research lies in using a neural lemmatization model as a proxy for linguistic change. The authors trained an LSTM-based lemmatizer on Mikael Agricola's works (considered early Old Literary Finnish) and then applied it to a larger corpus of Old Literary Finnish texts spanning from 1543 to 1809.
The key idea is that as the language evolves with new vocabulary, grammar, and spelling conventions, the lemmatization model, trained on earlier data, will encounter more errors. By analyzing the types and frequency of these errors across different decades, the researchers can infer the nature and timing of linguistic innovations. They also trained word embeddings on the lemmatized corpus to further explore semantic changes.
Datasets
- Old Literary Finnish Corpus (Institute for the Languages of Finland, 2013): This is the primary dataset used. It's described as the only proofread corpus of Old Literary Finnish, aiming for representativeness and containing approximately 4.13 million tokens.
- Agricola's Works: Used to train the initial lemmatization model. A morphosyntactically annotated version of Agricola's works was also available.
- Manually Lemmatized Dataset: The authors created a small, manually lemmatized dataset of 476 sentences from different decades to serve as a ground truth for evaluating the lemmatizer's performance and analyzing errors. This dataset was partially created using the Dictionary of Old Literary Finnish.
Key Insights and Ideas
- Errors as Linguistic Fingerprints: The most significant insight is that errors made by a language model trained on an earlier stage of a language can act as sensitive indicators of linguistic change in later texts.
- Gradual vs. Phased Change: The analysis suggests that linguistic change in Old Literary Finnish was more of a gradual process than a series of distinct, sharp phases, particularly concerning punctuation.
- Error Types Reveal Innovation: Different types of errors (e.g., misinterpreting new vocabulary, incorrect normalization of foreign words, or handling evolving punctuation) directly correlate with specific linguistic innovations.
- Potential for Automated Periodization: While the current corpus size and temporal distribution limit definitive periodization, the methodology demonstrates a promising path towards automating this process.
- NLP for Historical Linguistics: The study highlights the power of applying modern NLP techniques, even with limitations, to unlock insights from historical texts.
Key Results
- The word error rate (WER) of the lemmatization model on the Old Literary Finnish corpus varied between 1-23% and generally increased over time, indicating linguistic divergence.
- The model performed nearly flawlessly on early texts (Agricola's era) but showed increasing errors as it encountered newer linguistic features.
- Specific error categories identified included:
- Personal names: Difficulty with variations like "Ziphi" and "Zipheis."
- Emerging changes: Misinterpreting new pronoun forms (e.g., "sa" for "you") or new vocabulary.
- Multilinguality: The model struggled with Latin phrases embedded in Finnish texts.
- Punctuation and Conventions: Errors in handling evolving punctuation like commas and section signs (§), and abbreviations.
- Expanding Domains & Vocabulary: Difficulty with new terminology from scientific or modern concepts (e.g., "planets").
- The authors were able to trace the gradual emergence of punctuation conventions, such as the comma, and the increased use of hyphens.
Limitations and Future Work
- Corpus Temporal Distribution: The current corpus is not perfectly representative across all periods, with some decades having very little data, which limits definitive periodization.
- Data Size: The authors acknowledge that more data is needed to fully track actual semantic changes and to establish more robust periodization.
- Model Generalization: The model's performance is sensitive to unseen linguistic features, highlighting the challenge of generalizing to entirely novel vocabulary or grammatical structures.
- Future Work:
- Investigate whether the same model can be used with different languages or if mechanisms to ignore non-Finnish content can be implemented.
- Train models to automatically expand abbreviations.
- Develop more systematic methods for periodization, potentially by incorporating more data and refining error analysis.
- Improve the lemmatization model to better handle foreign words and neologisms.
- The authors have made their models and data openly available on Zenodo, encouraging further research and improvement.
Summary updated