Modeling Language Evolution in Historical NLP

This direction is split into other directions: diachronic semantic drift. Its last version stays readable and citable here.

Version 1 of 144 papers53 conceptsfrozen 2026-10-04

Most modern language models are "chronocentric." They are trained on massive scrapes of the contemporary internet, which makes them incredibly proficient at understanding 21st-century slang, technical jargon, and social norms. However, when these models are asked to process a 19th-century novel or an ancient Chinese poem, they often fail.

The reason is diachronic semantic shift: the tendency for word meanings, connotations, and even syntax to drift over time. To a modern model, "dinner" is an evening meal, but as Allen Kim and Charuta Pethe point out in What time is it? Temporal Analysis of Novels, in older literature, it often refers to a midday feast. Without accounting for this shift, AI misinterprets the very "wall-clock time" of historical narratives.

The Terminology Gap

The challenge isn't just that words change meaning; it’s that the concepts we use to search for information change as well. Yating Zhang and Adam Jatowt describe this as the "terminology gap" in Omnia Mutantur, Nihil Interit: Connecting Past with Present by Finding Corresponding Terms across Time. A researcher looking for the historical equivalent of an "iPod" in the 1980s needs to find a "Walkman."

Traditional keyword-based search fails here because there is zero literal overlap between the terms. Instead, researchers are developing semantic transformations that map concepts based on their functional roles within a specific era. This is particularly vital in fields like political science. Goran Glavaš and colleagues note in Political Text Scaling Meets Computational Semantics that an ideological term’s connotation in one decade may be completely inverted in the next, making it impossible to place political actors on a spectrum without a time-sensitive lens.

Teaching Models to Tell Time

To bridge this gap, researchers are moving away from static models toward "time-aware" architectures. One elegant approach is time masking. In Time Masking for Temporal Language Models, Guy Rosin and Kira Radinsky introduced TempoBERT, which prepends time tokens (like <2021>) to text. By forcing the model to predict these tokens, it learns to treat time as a fundamental dimension of context, much like it treats the words surrounding a masked term.

Similarly, Han Ren and Yafeng Ren developed TALM (Time-Aware Language Modeling for Historical Text Dating), which allows models to "sense" the era of a document even when no date is explicitly mentioned. This is crucial for the digital humanities, where dating undated archival fragments is a primary task.

Bridging the Diachronic Gap in Classical Languages

The problem is most acute in languages with thousands of years of history, such as Chinese. Junyi Xiang and Maofu Liu describe a 3,000-year "diachronic gap" between Classical and Modern Chinese in Exploiting Phonetics and Glyph Representation at Radical-level for Classical Chinese Understanding.

Because the meanings of characters have shifted so drastically, modern models often hallucinate meanings. To solve this, researchers are looking inside the characters. By decomposing characters into their visual "radicals" and phonetic components, models like RPGCM can anchor archaic meanings to stable structural features that haven't changed as much as the definitions themselves. Yuqing Li and Ruihua Song take this a step further in A Multi-Modal Knowledge Graph for Classical Chinese Poetry by mapping archaic words to "sememes"—universal, stable semantic units—to link ancient metaphors to modern visual imagery.

Why It Works: Cross-Temporal Alignment

The intuition behind these successes is alignment. Just as machine translation aligns English and French by finding shared structures in vector space, historical NLP aligns "English 1850" and "English 2024."

  1. Continued Pre-training: Ali Al-Laith and Daniel Hershcovich showed in Development and Evaluation of Pre-trained Language Models for Historical Danish and Norwegian Literary Texts that taking a modern model and giving it a "history lesson" through specialized corpora significantly improves its ability to handle sentiment and nuance in 19th-century texts.
  2. Language Pivoting: When historical data is scarce, researchers use high-resource modern languages as a bridge. Simone Tedeschi and Roberto Navigli demonstrated in Language Pivoting from Parallel Corpora for Word Sense Disambiguation of Historical Languages that English translations of Latin can be used to "pivot" modern semantic labels back onto ancient texts.

By treating time as a variable rather than a constant, these researchers are ensuring that AI doesn't just read the past through the biased lens of the present, but understands it on its own terms.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_modeling_language_evolution_in_historica_36f48a41,
  title = {Modeling Language Evolution in Historical NLP},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/modeling-language-evolution-in-historical-nlp/rdr_36f48a41670f4a0ba50d2edc367ed835},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_modeling_language_evolution_in_historica_36f48a41_v1,
  title = {Modeling Language Evolution in Historical NLP},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/modeling-language-evolution-in-historical-nlp/rdr_36f48a41670f4a0ba50d2edc367ed835/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}