Words change meaning on their own schedules. "Cell" picked up its telephone sense within a couple of decades. Grammatical shifts like the move of "actually" toward expressing the speaker's stance took much longer. Most computational work on semantic change still compares a word at exactly two time points and reports one number for how much it changed. That number cannot say when the change started, how fast it ran, or whether the word broadened and then narrowed again. We propose to treat each word's history as a time series of "change states" and align it to a small set of shared prototype shapes using dynamic time warping (DTW). DTW is an algorithm that finds the best stretch-and-compress mapping between two sequences, so that matching shapes line up even when they unfold at different speeds. The key move is to keep the warp rather than discard it. Aligning a word to a prototype splits its history into a shape (what kind of change it underwent) and a warp (when the change began and how fast it ran). The warp is then the measurement of interest.
This differs from the obvious use of DTW at this intersection, which is to cluster word trajectories while treating timing differences as nuisance. That use would erase exactly the distinction that human-annotated benchmarks care about. RuShiftEval separates "early shift and stabilize" words from "late bloomers," and those two classes have essentially the same shape at different times. A purely speed-invariant distance would merge them. Our claim is that shape and tempo should be estimated jointly and reported separately. Doing so gives dated, rate-aware descriptions of semantic change that two-period scores cannot provide.
Core Claim
Each word's diachronic trajectory can be represented in a word-independent "state of change" space. In that space, a few prototype shapes plus per-word monotone warps explain most trajectories. Once the prototypes are fitted, the warp yields onset year, duration and rate. We claim these estimates recover the true onset and rate of known changes, and correlate with dictionary first-attestation dates of new senses. We also claim they beat the simpler options on words whose histories are not a single S-shaped transition: change-point detection, sigmoid curve fitting, and a single diffusion rate per word.
Why this gap is real
Prior work plays three roles here. Some of it motivates the gap. Periti et al. argue that two-period comparison is an oversimplification, and that aligning results across many periods causes cluster misalignment and error propagation; they propose no algorithm. Hamilton et al. (EMNLP 2016) show that slow linguistic drift and abrupt cultural shift behave differently and are picked up by different measures. That is direct evidence that tempo is a variable worth measuring.
Other work supplies the representations but fixes the clock. Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change aligns decade-level embedding spaces with orthogonal rotations and measures displacement against calendar time. Dynamic Word Embeddings smooths trajectories under a Brownian-motion prior with one diffusion constant for the whole vocabulary. Dynamic Contextualized Word Embeddings uses a mean-reverting prior on a shared clock and needs social graphs that historical corpora lack. None of these compares two trajectories under different speeds. We reuse their per-period representations and add the alignment layer they lack.
A third group supplies the method. shapeDTW shows that matching windowed local descriptors instead of raw points prevents peaks from being matched to valleys; it beat standard DTW on 64 of 84 UCR datasets in 1-nearest-neighbour classification. We use the same idea to blunt sampling noise in sparse decades. Learning Discriminative Prototypes With Dynamic Time Warping and TimewarpVAE show that prototypes and warps can be learned jointly; we adapt that joint fit to linguistic trajectories. OTW gives a linear-time, metric alternative for one-dimensional probability measures. We use it as a speed variant for our one-dimensional displacement channel only, because its authors state that the multivariate extension is open.
Finally, some work constrains the design. Fittschen et al. show that modern language models fine-tuned on historical slices leak later senses into earlier periods. Any onset estimate built on such representations would be biased toward too-early dates. We therefore build representations only from period-local models. The finding from SemEval-2020 Task 1 that static embeddings beat BERT points the same way. Against it, UiO-UvA found that diachronically fine-tuned contextual models do better, so we run both representation families.
Research Program
Corpus. We build fine-grained trajectories from the Corpus of Historical American English, using roughly twenty decades from 1810 to 2000. For the OED-linked evaluation we add the 1750–1940 Gutenberg slices of Fittschen et al., subdivided into finer windows.
Representations. For each target word and each window we compute two things. The first is a static skip-gram vector, aligned to its neighbouring windows by orthogonal rotation as in Hamilton et al. The second is a distribution over induced senses: usages are embedded with a model trained only on that period or earlier, then clustered once over all periods, following the "unified clustering" option in Periti et al.
State-of-change descriptor. Sense inventories differ from word to word, so raw trajectories cannot be compared across words. We map each time step to a descriptor that has the same meaning for every word. It has five parts:
- the Jensen–Shannon distance of the current sense distribution from the earliest one;
- the Jensen–Shannon distance from the latest one;
- the entropy of the sense distribution;
- the overlap of the word's nearest-neighbour set with its initial neighbours, which is Hamilton's local measure;
- log frequency.
In the spirit of shapeDTW, each step's descriptor is concatenated over a short window of neighbouring steps. Each component carries a bootstrap variance estimate, so that sparse decades are down-weighted in the alignment cost. This is aimed directly at the "ambiguity trap" that RuShiftEval identifies, where small samples make sense proportions look as if they shifted when they did not.
Model. We fit $K$ prototype trajectories $p_1,\dots,p_K$ and, for each word $w$, an assignment $k_w$ and a monotone warp $\pi_w$ from calendar time to prototype time. The objective is
$$\min_{\{p_k\},\{k_w,\pi_w\}} \sum_w \Big[\sum_t \tfrac{1}{\sigma^2_{w,t}}\, \| x_{w,t} - p_{k_w}(\pi_w(t)) \|^2 + \lambda\, R(\pi_w)\Big].$$
Here $x_{w,t}$ is the descriptor of word $w$ at calendar step $t$, and $\sigma^2_{w,t}$ is its bootstrap variance. $R$ penalises warps that deviate from a constant slope, and $\lambda$ sets how much elasticity is allowed. The fit alternates between DTW (inside a Sakoe–Chiba band, which limits how far the alignment may stray from the diagonal) for the warps and assignments, and barycentric updates for the prototypes. This is the same alternating scheme that the dtgw work uses to make a hard joint problem tractable.
Reading the output. Each prototype is annotated with its transition region. A word's onset is the calendar year that maps to the start of that region, and its rate is the inverse slope of the warp across it. Multi-stage prototypes, such as broadening followed by narrowing, give one onset and one rate per stage.
Cross-corpus lag. As a secondary use, we warp the same word's trajectory in two genres directly against each other, fiction against magazines in COHA. The warp offset estimates how far one genre lags the other.
Baselines
Do-nothing reference. The naive reference is the endpoint score that the field reports today: the Jensen–Shannon distance between the first and last period. For dating tasks, its equivalent is predicting the corpus midpoint as every word's onset. Our method must clearly beat both, or the extra machinery is pointless.
Strongest existing method for graded change. On graded change ranking, the strongest cited competitors are the systems in UWB at SemEval-2020 and UiO-UvA. UWB uses skip-gram embeddings aligned by canonical correlation analysis or orthogonal mapping. Its 4th-place Subtask 2 average Spearman correlation is 0.481, and it took 1st on Subtask 1 with accuracy 0.687. UiO-UvA, which fine-tunes contextual models on the diachronic corpora, was the top Subtask 2 submission; the file quotes only relative gains for it, not an absolute figure. We rerun UWB's pipeline on the same English data. If the method reports nothing new about endpoints, our endpoint-derived score should not fall below it. This is a check that adding trajectories costs nothing, not the headline result.
Strongest competitor on the actual question. On dating and rate, the strongest competitor is not a published semantic-change system, since none of them dates change. It is the obvious statistical tool: fit a logistic S-curve to each word's displacement-from-start curve, and separately run Bayesian change-point detection on the same curve. We also run the Dynamic Word Embeddings smoothing model, modified so each word has its own diffusion constant, which gives a rate but no onset. All competitors consume the same per-window representations and the same bootstrap variances. Each receives the same hyperparameter search budget, tuned on the same held-out synthetic words. Beating them means lower onset error and higher rate correlation on multi-stage and noisy words. It also means no worse than them on single-transition words, with a 95% confidence interval from bootstrap over words that excludes zero.
Ablations. Three ablations isolate the key ingredients:
- Rigid prototype fit. Setting $\lambda\to\infty$ forces a constant-slope warp (an offset only), so the prototypes stay but elasticity is removed.
- Pointwise descriptor. Dropping the windowed descriptor and the variance weighting removes the shapeDTW component.
- Speed-invariant DTW clustering. Clustering with plain DTW and discarding the warps is the "obvious idea." We predict it collapses the RuShiftEval early-shift and late-bloomer classes.
Why not just…?
Why not just fit an S-curve to each word? Language change is often S-shaped, and for a word with one clean transition a sigmoid fit will probably date the change about as well as our method does. We treat this as a serious threat, not a straw man. The difference should appear where a sigmoid is the wrong model. Broadening-then-narrowing produces a hump rather than an S, sense branching is visible only in the entropy channel, and sparse decades make a three-parameter fit jumpy. In the synthetic experiment the results are stratified by trajectory family. If sigmoid fitting matches us on the multi-stage families too, the conclusion is that prototypes add nothing, and we will report that.
Why not just run change-point detection? Change-point detection gives an onset but no rate. It also gives no notion of which words share a kind of change, and one false break from sampling noise is enough to move its estimate. Our variance-weighted alignment pools evidence across all words that share a prototype. That pooling should matter most for low-frequency words. The frequency-stratified results of the synthetic experiment test this directly.
Why not just give each word its own diffusion constant in a dynamic embedding model? A per-word rate in the style of Dynamic Word Embeddings captures how much a word wanders. It does not capture when, and it cannot tell a word that drifts steadily from one that is dormant and then jumps, the distinction Hamilton et al. (EMNLP 2016) draw. We include this model as a baseline for rate correlation. It is expected to be competitive on steadily drifting words and weak on bursty ones.
Why not just cluster trajectories with Euclidean distance after normalisation? Rigid distance compares a word in 1900 with its prototype in 1900. Two words with the same change forty years apart then look dissimilar, and their prototypes smear into flat averages. This is exactly the rigid-warp ablation, so the experiments measure it directly.
Proposed Experiments
Synthetic pseudo-word benchmark. This is the decisive test because ground truth is exact. We build pseudo-words by merging two real low-change words in COHA. The proportion of usages drawn from the second word follows a schedule we control: sigmoid, linear, step, hump (gain then loss), or two successive stages. Onsets and durations vary, and frequencies range across three orders of magnitude. The metrics are absolute onset error in years, Spearman correlation between estimated and true rates, and prototype-family accuracy, all reported per family and per frequency band. The method succeeds only if it matches the sigmoid and change-point baselines on single sigmoid transitions and beats them on hump and two-stage schedules. It must also beat the rigid-warp ablation everywhere that onsets vary. If it does not beat the baselines on multi-stage schedules, the core claim fails.
OED first-attestation dating. On the OED cloze data of Fittschen et al., we correlate each word's estimated onset with the attested date of its new sense. Representations come from their period-isolated models and, separately, from a fine-tuned modern model. We expect the leaky representation to bias onsets earlier, and measuring the size of that bias is itself a result. No reported dating result was found on Lacuna for this dataset, so the baselines above are our reference points.
RuShiftEval trajectory types. We test whether warp-derived onsets separate continuous-change, early-shift and late-bloomer words, while plain DTW clustering does not. We will reconstruct finer within-era windows if the underlying corpus permits. Otherwise the test uses only the three periods, which caps how much warping can show, and we will state that limit. The file reports no model scores for the trajectory types, beyond annotator agreement of about 0.5–0.6 Krippendorff's α, so the comparison is internal to our baselines.
SemEval English endpoint check. Here the endpoint Jensen–Shannon distance from our sense distributions is compared with UWB's reported 0.481 average Spearman correlation. The aim is to show that building trajectories does not degrade endpoint ranking.
Genre lag. Cross-genre warps between COHA fiction and magazines are validated first on synthetic pseudo-words with an imposed lag. They are then reported descriptively for the known cultural-shift nouns that Hamilton et al. discuss. This part is exploratory.
References
- Three-part diachronic semantic change dataset for Russian
- Towards a Complete Solution to Lexical Semantic Change: an Extension to Multiple Time Periods and Diachronic Word Sense Induction
- Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Change
- Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
- Dynamic Word Embeddings
- Dynamic Contextualized Word Embeddings
- shapeDTW: shape Dynamic Time Warping
- Learning Discriminative Prototypes With Dynamic Time Warping
- TimewarpVAE: Simultaneous Time-Warping and Representation Learning of Trajectories
- OTW: Optimal Transport Warping for Time Series
- Pretraining Language Models for Diachronic Linguistic Change Discovery
- SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection
- UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detection
- Comparing temporal graphs using dynamic time warping
- UWB at SemEval-2020 Task 1: Lexical Semantic Change Detection