TL;DR
This paper introduces a novel computational method to automatically track how word combinations (collocations) change over time in historical texts. By analyzing syntactic relationships and statistical associations, the system identifies lost collocations and suggests modern replacements, offering a valuable tool for historical linguists.
What Problem Does This Paper Address?
Natural language is constantly evolving, and this change is evident in how words combine. While historical linguists have long studied these diachronic (over time) shifts, manually sifting through vast historical corpora to identify changes in collocations is a laborious and time-consuming task. Existing computational methods for tracking language change often focus on individual words or broader semantic shifts, but there's a need for tools specifically designed to pinpoint changes in the way words are paired together. This paper aims to fill that gap by providing an automated system to detect and analyze these collocational variations.
Methods
The proposed method combines several techniques:
- Dependency Parsing: The system first uses dependency parsing to identify syntactically related pairs of words (head-dependent relationships) within historical texts. This helps to extract potential collocation candidates.
- Statistical Association Measures: These candidates are then ranked using statistical association measures (like log-likelihood) to identify those that occur together more frequently than by chance, distinguishing them from random word co-occurrences.
- Diachronic Tracking: The core of the method involves tracking these identified collocations across different time periods of a corpus. For each collocation, the system determines:
- When it first appears.
- Whether it disappears in later periods.
- If it disappears, the type of change it underwent (e.g., loss of one or both words, or loss of the specific combination).
- Distributional Semantics for Replacements: When a collocation is no longer used, the system leverages distributional semantics (specifically, word embeddings) to suggest potential modern replacements. This involves finding words or phrases that convey similar meanings to the lost collocation.
- Frequency Trend Analysis: For collocations that continue to be used, the system analyzes their frequency trends, classifying them as increasing, decreasing, or stable.
Datasets
The authors conducted a case study using historical corpora of Portuguese and Spanish from the P.S. Post Scriptum project. These corpora cover the 16th to 19th centuries and are provided in a CoNLL-like format. For contemporary resources, they used dictionaries and Wikipedia data for both languages. For training word embeddings, they used a combination of historical and contemporary texts, including "cuentos españoles," "romances españoles," and the "Colonia historical corpus" for Portuguese. They also tested their system on a non-normalized historical corpus of Portuguese ("Colonia").
Key Insights and Ideas
- Collocations are dynamic: The study highlights that collocations, like individual words, are subject to significant change over time, with some disappearing and others emerging or shifting in meaning.
- Automated detection is feasible: The paper demonstrates that computational methods can effectively automate the identification of these diachronic changes, significantly speeding up the work of historical linguists.
- Distributional semantics as a bridge: Using word embeddings to find semantic replacements for lost collocations is a powerful way to connect historical language use with contemporary language.
- Different types of change: The system categorizes changes into distinct types (loss of one or both lemmas, disappearance of the collocation), providing a structured way to understand language evolution.
- Normalized vs. Non-normalized Corpora: The system shows promise even with non-normalized historical texts, although normalization generally improves performance.
Key Results
The quantitative evaluation on Portuguese and Spanish corpora identified:
- 1,932 and 1,980 changes of types 1-4 (loss of collocations or their components) in Spanish and Portuguese, respectively.
- Approximately 90% of these changes were type 4 (loss of the specific combination while lemmas remain).
- Historical trends (changes 5-7) were extracted for 3,129 (Spanish) and 2,210 (Portuguese) combinations.
- The system achieved a precision of 52.1% (overall) for Spanish and 68.8% (overall) for Portuguese.
- Qualitative analysis revealed interesting cases of lost collocations and their suggested replacements, such as the Portuguese "deitar missa" (to say mass) being lost and the Spanish "prestar paciencia" (to lend patience) being replaced by "tener paciencia" (to have patience).
Limitations and Future Work
The authors acknowledge several limitations and suggest future directions:
- Parsing Errors: The system's performance is affected by errors in the dependency parser, particularly with non-normalized corpora. Future work aims to improve preprocessing for such texts.
- Distributional Semantics Limitations: Standard distributional semantics models can struggle to capture the specific senses of collocates, leading to incorrect suggestions. The authors propose exploring non-compositional methods or contextualized compositional strategies.
- Gold Standard Data: The lack of a gold standard for diachronic collocation variation makes systematic evaluation challenging.
- Future Work:
- Improving NLP tools for non-normalized corpora.
- Developing more balanced word embedding models trained on historical resources.
- Utilizing contextualized distributional semantics models to better infer word senses based on context.
- Integrating the system into a visualization tool to aid historical linguists and digital humanities researchers.
Summary updated