TL;DR
This paper introduces a novel computational method for studying how concepts, rather than just individual words, change their meaning over time. By automatically inducing concepts from historical text data, it enables a more systematic, "concept-to-word" perspective on lexical semantic change, complementing existing "word-to-concept" approaches.
What Problem Does This Paper Address?
Lexical Semantic Change (LSC) is the study of how word meanings evolve. While computational linguistics has made strides in tracking changes in individual words (semasiology), there's a gap in understanding how the concepts themselves are expressed differently over time (onomasiology). Existing NLP methods primarily focus on specific target words and their sense inventories, overlooking the broader conceptual landscape. This paper aims to bridge this gap by developing a method to automatically discover and track concepts from historical texts, allowing for a more holistic view of semantic evolution.
Methods
The core of the paper is a two-step hierarchical clustering approach to induce concepts from word occurrences in a diachronic corpus.
- Lemma-Centric Clustering (Algorithm A₁): For each individual lemma (word form) within a specific time period, its contextualized word embeddings are clustered to identify different "sense clusters." This is similar to traditional Word Sense Induction but aims to group occurrences that share a concept, even if they come from different lemmas.
- Cross-Lexicon Clustering (Algorithm A₂): The "sense clusters" from all lemmas across all time periods are then further clustered. This step merges sense clusters that represent the same underlying concept, regardless of the specific lemmas they originated from. The output is a set of "concept clusters."
The authors use XLM-R (a multilingual transformer model) to generate contextualized word embeddings, leveraging its zero-shot cross-lingual capabilities for French. They also propose a heuristic for selecting hyperparameters: maximizing the number of concept clusters containing 2-5 lemmas in the most recent time period, as larger clusters tend to be noisy.
Datasets
The primary dataset used is the Presto Corpus, a historical French corpus spanning from 1500 to 1950. The authors selected 623 frequent noun lemmas from this corpus, focusing on those appearing in at least three consecutive 50-year spans and at least 10 times overall. For their analysis, they divided the corpus into three broad time periods: 1500-1699, 1700-1799, and 1800-1949.
Key Insights and Ideas
- Onomasiology is Underexplored in NLP: The paper highlights that while historical linguists often adopt an onomasiological perspective, NLP research has heavily favored the semasiological view. This work demonstrates a computational path to address the onomasiological side.
- Concepts as Clusters of Lemmas: The paper defines a concept as a set of lemmas used to express it. This operationalization allows for computational tracking of conceptual evolution.
- Polysemy and Semantic Change are Linked: The study finds a significant positive correlation between the number of senses a lemma has and its degree of semantic change (measured by Jensen-Shannon Divergence between sense distributions across time). Lemmas with more senses are more prone to gaining or losing senses.
- Conceptual Stability vs. Naming Evolution: While 86% of induced concept clusters maintained an "identical naming" (i.e., the set of lemmas representing the concept remained largely the same) over time, the paper also identifies instances of "expanded naming" (new lemmas joining a concept) and "shrinked naming" (lemmas leaving a concept). These changes can reveal interesting patterns, such as the borrowing of concrete terms for abstract concepts.
Key Results
- The methodology successfully induces concept clusters from a diachronic corpus.
- A qualitative evaluation of small concept clusters (2-4 lemmas) in the last time period found that only 10% were invalid, with 30% being actual synonyms and 20% near-synonyms.
- The study confirms a positive correlation between lemma polysemy and semantic change.
- Analysis of concept-cluster naming evolution revealed that most concepts remain stable in their lexical realization, but some undergo expansion or shrinkage of their associated lemmas, offering insights into how concepts are expressed over time.
Limitations and Future Work
- Lack of Sense-Annotated Data: The authors acknowledge that without sense-annotated diachronic data, it's difficult to definitively evaluate the quality of the induced concept clusters and the precision of the clustering at the occurrence level.
- Heuristic for Hyperparameters: The selection of hyperparameters is based on a heuristic (maximizing clusters of 2-5 lemmas) due to the absence of annotated data. Future work could explore different heuristics or unsupervised evaluation metrics.
- Lemma-Centric Clustering Noise: The lemma-centric clustering might sometimes group word usages instead of actual word meanings, potentially leading to clusters that represent lexical/topical relations rather than pure synonyms. Improving this step is a direction for future research.
- Corpus Sparsity: The relatively small size and sparse nature of the Presto Corpus limit detailed analysis and fine-grained results. The authors suggest that larger time periods were necessary to balance lemma representation, but this sacrifices the ability to detect subtle or short-lived semantic changes.
- Invalid Clusters: The approach can group together words tokenized into multiple subwords without a clear semantic relation. Further research is needed to parse these invalid clusters into more meaningful ones.
Summary updated