Towards a Complete Solution to Lexical Semantic Change: an Extension to Multiple Time Periods and Diachronic Word Sense Induction

Francesco Periti, Nina TahmasebiACL 2024

TL;DR

This paper argues that current methods for tracking how word meanings change over time are too simplistic, often only considering two time periods. The authors propose extending these methods to handle multiple time periods and diachronic word sense induction, highlighting the computational and interpretability challenges involved. They outline several approaches for more robust semantic change modeling, aiming to better support linguists and historians.

What Problem Does This Paper Address?

Lexical Semantic Change (LSC) is the study of how word meanings evolve over time. While the field has made progress, current computational approaches typically simplify the problem by focusing on just two time periods (t1 and t2). This simplification is largely due to the computational challenges of handling large historical corpora and the lack of comprehensive benchmarks for multiple time periods. The authors argue that this two-period approach is insufficient for truly understanding the nuances of semantic change and the evolution of word senses. They propose moving towards methods that can handle multiple time periods and perform "diachronic word sense induction" – identifying distinct word senses across historical periods. This is crucial for tasks like understanding how a word's meaning broadens, narrows, or develops entirely new senses over extended historical durations.

Methods

The paper doesn't present new algorithms but rather discusses extensions and challenges in existing LSC methodologies. The core idea is to move beyond pairwise comparisons (t1 vs. t2) to a more continuous, multi-period analysis. Key methodological discussions include:

  • Clustering over Consecutive Time Intervals: Re-executing pairwise clustering methods sequentially (t1-t2, then t2-t3, etc.) and using cluster alignment to link results. This is computationally intensive and can lead to inconsistencies.
  • Clustering over Consecutive Time Periods: Clustering embeddings for each time period individually and then aligning clusters across periods. This is more scalable per clustering step but still requires significant alignment effort.
  • One-Time Clustering over All Time Periods: A single clustering operation on embeddings from all time periods. This is computationally feasible but can obscure fine-grained temporal changes.
  • Incremental Clustering: Algorithms designed to update clustering results as new data arrives, making them suitable for dynamic, evolving corpora. This approach aims to maintain a "memory" of past word meanings.
  • Form-based Approaches: Methods that focus on word form (e.g., word co-occurrence patterns) rather than explicit sense induction. These are proposed as a way to efficiently monitor stability and identify potential "change points" in the data, signaling where more detailed sense-based analysis might be needed.

Datasets

The paper does not introduce new datasets. It discusses the need for better benchmarks for multi-period LSC, noting that existing benchmarks are often limited to two time periods and may lack diachronic sense labels.

Key Insights and Ideas

  • The "Two-Period" Simplification is a Bottleneck: The current standard of analyzing LSC between just two time points is a significant oversimplification that hinders a deeper understanding of semantic evolution.
  • Diachronic Word Sense Induction is Crucial: To truly model semantic change, we need to identify and track distinct word senses across multiple historical periods, not just aggregate all usages.
  • Computational and Interpretability Challenges Escalate with Time Periods: As the number of time periods increases, the computational burden (e.g., sampling, GPU usage) and the complexity of interpreting automatically derived clusters into meaningful sense inventories grow dramatically.
  • Incremental Methods Offer Promise: For modeling dynamic language evolution, incremental clustering approaches are highlighted as a more suitable direction than traditional batch clustering.
  • Form-based Methods Can Guide Sense-based Analysis: Using form-based methods to detect periods of stability or change can help focus more computationally expensive sense-based analyses on the most relevant time intervals.

Key Results

As this is a position paper, it does not present empirical results or quantitative findings. Instead, it outlines conceptual challenges and proposes directions for future research.

Limitations and Future Work

  • No New Algorithms or Experiments: The paper is a position paper and does not introduce novel algorithms or report experimental results.
  • Focus on Unsupervised Methods: The discussion primarily centers on unsupervised methods that induce senses through clustering of word representations (specifically contextualized embeddings). Other methods like word masking or static embeddings are mentioned but not deeply explored in the context of the proposed extensions.
  • Detecting Change Type and Cause: The paper acknowledges that detecting the type of semantic change (e.g., broadening, narrowing) and its cause are important areas not fully addressed, though the proposed methods could inherently support type detection.
  • Future Work: The authors suggest that future work should focus on developing methods for aligning similar clusters and detecting change types to speed up the interpretation process. They also emphasize the need for better ways to frame and store diachronic sense inventories, requiring collaboration between computational linguists and text-based researchers.

Summary updated

BibTeX
@inproceedings{periti2024towards,
  author = {Francesco Periti and Nina Tahmasebi},
  title = {{Towards a Complete Solution to Lexical Semantic Change: an Extension to Multiple Time Periods and Diachronic Word Sense Induction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure