The Rationality of Semantic Change

Omer KoratACL 2019

TL;DR

This paper proposes that semantic change in function words is driven by a principle of contrast, where newly adopted words in a syntactic environment push out older, competing words. By analyzing historical corpora with syntactic information, the study demonstrates that this competition leads to predictable divergence in word usage over time, enhancing communicative efficiency.

What Problem Does This Paper Address?

The paper tackles the question of why and how the meaning of words changes over time, a phenomenon known as semantic change. While much research has focused on content words and their semantic shifts, the dynamics of function words (like prepositions and discourse markers) have been less explored. A key challenge is that function words often have multiple syntactic roles, and their meaning can be highly dependent on context. Traditional methods, like bag-of-words models, struggle to capture these nuanced distinctions, leading to an incomplete understanding of semantic change. This paper aims to bridge this gap by investigating how function words compete and diverge in their usage within specific syntactic environments, driven by a principle of rational communication.

Methods

The core of this research lies in its novel approach to analyzing historical language data. Instead of relying on simple word co-occurrence, the authors utilize manually parsed historical corpora. This allows them to go beyond a "bag-of-words" representation and capture the syntactic position of words.

Key methodological aspects include:

  • Syntax-based representations: Words are analyzed not just by what they appear with, but where they appear in a sentence structure. This enables distinguishing between different uses of the same function word (e.g., "so" as a complementizer vs. "so" as a manner adverb).
  • Principle of Contrast (relaxed version): The study operationalizes a principle that suggests words competing for a similar meaning in a given environment will tend to diverge in their usage patterns. This is framed as a rational strategy for speakers to optimize communicative efficiency.
  • Quantitative analysis of distributional change: The authors measure semantic change by tracking the relative frequencies of competing word uses across different syntactic environments over time. They use statistical models (specifically, cubic models with splines) to analyze these trends and identify points where one word's frequency increases while another's decreases.

Datasets

The study uses two primary datasets:

  • Penn Parsed Corpora of Early Modern English (Kroch and Delfs, 2004): This corpus provides syntactically parsed data for a historical period.
  • Parsed Corpora of Early English Correspondence (Taylor and Nevalainen, 2006): Another syntactically annotated corpus for historical language analysis.

These corpora are crucial because they are manually annotated, allowing for the detailed syntactic analysis that is central to the paper's methodology. The paper does not introduce new benchmark datasets but leverages existing, rich resources.

Key Insights and Ideas

  • Function words are rational communicators: The paper argues that function words, like content words, are subject to principles of communicative efficiency. When multiple words can serve a similar function in a specific syntactic slot, speakers will naturally favor one over the other, leading to divergence.
  • Syntax is key to understanding semantic change: The ability to distinguish between different syntactic uses of a word is crucial for understanding its semantic evolution. Bag-of-words models, which flatten all uses into a single representation, miss this critical dimension.
  • The Principle of Contrast drives divergence: When a word "immigrates" to a new syntactic environment where it competes with existing words, the principle of contrast predicts that the older words will become less frequent in that specific environment. This is not about words disappearing entirely, but about them being "pushed" to other niches.
  • Semantic change is not random: The observed patterns of divergence suggest a rational basis for semantic change, aiming to reduce ambiguity and cognitive load for speakers and listeners.

Key Results

The study found significant evidence supporting the hypothesis that competing word uses diverge over time. Specifically:

  • Opposite trends observed: For most competing pairs analyzed (e.g., "hence" vs. "therefore," "but" vs. "except," "as" vs. "when"), the study found that as one word's frequency increased in a particular syntactic environment, the competing word's frequency decreased.
  • Statistical significance: The statistical models used to capture these trends were largely significant (p < .05), indicating that the observed changes were not due to random chance.
  • Limited impact on co-occurrence: Interestingly, the study found that semantically similar word pairs did not tend to diverge in their co-occurrence patterns (i.e., the words they appear with in the same sentence). This reinforces the idea that the divergence is driven by structural/syntactic competition rather than broader semantic similarity.
  • No complete disappearance: A notable finding is that none of the analyzed word uses completely disappeared, suggesting that competition leads to specialization rather than extinction.

Limitations and Future Work

The authors acknowledge several limitations:

  • Narrow empirical coverage: The study is limited to a "closed set of word pairs" that could be reliably distinguished in the annotated corpora. This means the findings, while strong for the examined cases, may not generalize to the entire vocabulary without further work.
  • Small r² values: In some cases, the statistical models explained only a small portion of the variation in the data. This suggests that time is not the sole driver of frequency changes; other factors likely play a role.

Future work could include:

  • Automated detection of competing environments: Developing methods to automatically identify syntactic environments where word pairs compete would allow for a broader analysis across the entire vocabulary.
  • Extending models to account for complete use: Further research could explore how to extend Rational Speech Acts (RSA) or game-theoretic models to explain why competing uses don't disappear entirely, perhaps by considering how distinctions are maintained (e.g., through marking subtle social or attitudinal information).
  • Investigating other functional word categories: Applying the methodology to other types of function words could provide a more comprehensive understanding of semantic change.

Summary updated

BibTeX
@inproceedings{korat2019rationality,
  author = {Omer Korat},
  title = {{The Rationality of Semantic Change}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure