Modeling Diachronic Semantic Drift in Historical Texts

41 concepts35 papers1 proposal

Language is not a static repository of meanings; it is a living system that evolves alongside culture, technology, and social norms. When we use modern natural language processing (NLP) tools to analyze historical archives, we encounter a fundamental obstacle known as the terminology gap. A search for "iPod" in a 1980s archive will fail to return relevant results about the "Walkman," not because the concept of a portable music player is missing, but because the lexical label has shifted.

This phenomenon, known as diachronic semantic drift, describes how the meanings, connotations, and usage patterns of words change over decades or centuries. As Berberich et al. argue, bridging this gap requires more than simple keyword mapping; it requires models that can reformulate modern queries into the specific linguistic "flavor" of a target era.

The Problem of Temporal Blindness

Most modern large language models (LLMs) are trained on contemporary snapshots of the web, leading to what Hegde et al. describe as "temporal blindness." These models often apply modern linguistic norms to historical texts, leading to significant misinterpretations.

The drift is rarely just a matter of new inventions. It often involves subtle shifts in everyday concepts. For example, Kim et al. demonstrate that the word "dinner" functioned as a temporal anchor that shifted from midday to evening across different historical eras. A model that does not account for this diachronic change will misidentify the time of day in 19th-century literature. Similarly, Li found that color adjectives in fiction have drifted from describing abstract emotions (like "blue" for sadness) toward more concrete physical descriptions, a shift that modern embeddings may fail to capture accurately.

The Alignment Challenge

To track these changes computationally, researchers often use word embeddings to represent different time periods and then "align" these vector spaces. The intuition is that if most words remain stable, we can use them as anchors to rotate one time period's vector space to match another. Once aligned, any word that has moved significantly relative to the anchors is identified as having undergone semantic shift.

However, this process faces a circularity problem. Gruppi et al. point out that if the words chosen as anchors have themselves undergone subtle shifts, the entire alignment becomes skewed, effectively masking the very evolution the researcher is trying to detect. Furthermore, Kiyama and Aida note that models struggle to distinguish between permanent semantic shifts (a word's core meaning changing) and temporary topical shifts (a word appearing in new contexts due to a short-lived political event).

Bridging the Diachronic Gap

To overcome these hurdles, researchers have proposed several specialized architectures:

  • Time-Aware Modeling: Instead of treating time as a metadata tag, models like TALM (Time-Aware Language Model) proposed by Ren et al. and the framework by Chang et al. treat the historical era as a core feature. By predicting both the modern translation and the historical era simultaneously, these models learn to "sense" the chronological context of a text.
  • Continued Pre-training: Rather than building models from scratch, Al-Laith et al. show that modern models can be given a "history lesson" through continued pre-training on specialized historical corpora, significantly improving their performance on tasks like sentiment analysis in 19th-century texts.
  • Sub-lexical Features: For languages like Ancient Chinese, where characters may remain the same while meanings drift over 3,000 years, Xiang and Liu suggest looking at "radicals"—the visual and phonetic components of characters—to bridge the diachronic gap.

The Scarcity of Ground Truth

A recurring theme across this research is the difficulty of evaluation. As Bizzoni and Klakow highlight, there is a severe lack of "diachronic gold standards"—historical dictionaries or ontologies that can verify if a model's detected shift is accurate. Relying on modern lexical databases like WordNet to evaluate historical models is inherently flawed because it ignores the very semantic shifts the models are designed to find.

To address this, researchers are developing multi-era datasets like RuShiftEval, which Pivovarova explains allows for the mapping of complex "semantic trajectories" across three distinct sociopolitical eras rather than simple binary change detection. By moving toward these longitudinal frameworks, the field aims to transform NLP from a tool for the present into a lens for understanding the deep history of human thought.

Key Foundational Work

A Cross-Language Approach to Historic Document Retrieval (2007) — contributed 1 concept elements to this direction.
Bridging the Terminology Gap in Web Archive Search (2009) — contributed 1 concept elements to this direction.
A Probabilistic Approach to Language Change (2007) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_diachronic_semantic_drift_b1cab818,
  title = {diachronic semantic drift},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/diachronic-semantic-drift/rdr_b1cab81880fe4b5d976526bdc47e351d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_diachronic_semantic_drift_b1cab818_v1,
  title = {diachronic semantic drift},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/diachronic-semantic-drift/rdr_b1cab81880fe4b5d976526bdc47e351d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake