Studying Semantic Chain Shifts with Word2Vec: FOOD>MEAT>FLESH

Richard ZimmermannACL 2019

TL;DR

This paper uses Word2Vec to track the historical semantic shift of "food" from a general term to "meat" and then to "flesh" in English. By analyzing word embeddings across different time periods, the study provides quantitative evidence for a "push chain" semantic change, where the meaning of "food" influenced "meat," which in turn influenced "flesh."

What Problem Does This Paper Address?

Understanding how word meanings evolve over time, known as semantic change, is a fundamental challenge in linguistics. Semantic chain shifts, where a series of related words undergo meaning changes, are particularly complex to study manually due to the sheer volume of historical text and the subtle nature of semantic nuances. This paper specifically investigates the historical semantic shift of the lexical field related to "food" in English, focusing on the progression from a general term for sustenance to "meat" (animal flesh for consumption) and "flesh" (animal tissue, often not for consumption). The authors aim to determine the temporal course of this shift and whether it originated from "food" (a "push chain") or was driven by changes in "flesh" (a "pull chain").

Methods

The core methodology employed is the use of Word2Vec, a popular neural network-based word embedding technique. Word2Vec learns vector representations of words based on their surrounding context in a corpus. Words that appear in similar contexts are mapped to nearby points in a high-dimensional vector space.

The authors trained Word2Vec models on nine distinct 50-year periods spanning from 1425 to 1925. They utilized a continuous bag-of-words (CBOW) architecture with specific parameters: a vector size of 250, a context window of 20 words, and a minimum word count of 5.

To analyze the semantic shifts, they calculated cosine similarities between the vector representations of the target words (FOOD, MEAT, FLESH) and their neighbors across different time periods. This allowed them to quantify how closely related these words became over time. Additionally, they used Principal Component Analysis (PCA) to visualize the semantic relationships of the target words and their nearest neighbors in a 2D space, employing a Procrustes transformation to align the semantic spaces of different time periods.

Datasets

The study utilizes four historical corpora:

  • Innsbruck Corpus of Middle English Prose: Used for the earliest period (1425-1475).
  • Early English Books Online (EEBO): Used for periods from 1475-1725.
  • Eighteenth Century Collections Online (ECCO): Used for periods from 1676-1825.
  • Corpus of Late Modern English Texts (CLMET3.0): Used for periods from 1776-1925.

The total dataset comprises approximately 845 million words. The material was pre-processed to normalize spelling variations and lemmatize key lexemes to ensure consistency for the Word2Vec models.

Key Insights and Ideas

  • Quantitative evidence for semantic chain shifts: Word embeddings provide a powerful tool to move beyond anecdotal evidence and quantitatively track semantic evolution.
  • "Push chain" hypothesis supported: The data strongly suggests that the semantic shift originated with "FOOD" becoming more general, influencing "MEAT," which then influenced "FLESH."
  • Temporal dynamics of change: The study reveals distinct temporal phases for each word's semantic evolution, with "FOOD" showing changes earlier than "MEAT" and "FLESH."
  • Visualization of semantic space: PCA plots offer an intuitive way to visualize how the semantic neighborhoods of words change and interact over centuries.
  • Challenges in historical NLP: The paper highlights the significant challenges of working with historical texts, including spelling variation, genre effects, and the need for careful pre-processing.

Key Results

  • The cosine similarity between FOOD and MEAT increased steadily from around 0.4 in 1450 to 0.6 in 1700, remaining stable thereafter.
  • The cosine similarity between MEAT and FLESH showed a more gradual increase, peaking around 0.6 between 1700 and 1800, before diverging slightly by 1900.
  • These patterns align with a push chain interpretation, where "FOOD" initiated the changes.
  • The analysis of nearest neighbors in PCA plots shows "FOOD" dissociating from "sustenance" early on, while "MEAT" and "FLESH" gradually shifted their semantic associations.
  • The paper provides examples of conservative and innovative uses of the target words from different historical periods, illustrating the semantic shifts.

Limitations and Future Work

  • Periodization granularity: The 50-year periods might not be fine-grained enough to definitively establish the exact sequence of changes between "MEAT" and "FLESH."
  • Modern period analysis: The study's temporal scope ends in 1925, leaving the potential for ongoing semantic evolution in the last century unexamined.
  • Context word stability: Finding context words that maintain consistent meaning across such vast historical spans is challenging and can introduce distortions in dimensionality reduction.
  • Genre and social class influence: Word embeddings, in their current form, struggle to isolate the impact of specific social classes or genres on semantic change.
  • Future work suggestions:
    • Investigate the last century (1900-2000) for further semantic evolution.
  • Explore similar chain shifts in other languages (e.g., French).
  • Compare Word2Vec results with other methods like inter-annotator agreement studies or collocation analysis.
  • Further research into the influence of genre and social class on semantic change.

Summary updated

BibTeX
@inproceedings{zimmermann2019studying,
  author = {Richard Zimmermann},
  title = {{Studying Semantic Chain Shifts with Word2Vec: FOOD>MEAT>FLESH}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure