Timescale Spectra for Forecasting Temporal Knowledge Graphs

A temporal knowledge graph is a database of time-stamped facts such as "Country A threatened Country B on 3 March." Forecasting means predicting which facts will appear at a future date the model has never seen. The most robust forecasters describe time relatively. They ask how long ago a related fact happened, not on what calendar date. Elapsed times recur in the future even when calendar dates do not. Current systems, however, use one fixed shape for how quickly an old fact loses relevance. INFER applies the same square-root decay to every relation. TimeTraveler uses a single learned function of elapsed time. Real relations do not behave this uniformly. A diplomatic visit may predict another visit within days and also echo an annual summit cycle. An alliance may stay predictive for years.

We propose to give every relation a timescale spectrum: a small, learned set of weights over a fixed bank of decay rates, log-spaced from hours to decades. Each weight says how much evidence from that timescale matters. The bank also includes an infinite-timescale band, which never decays and is therefore simply a static, time-free compatibility score. Including it folds the "facts never seen before" case into the same model as recurring facts. That case is large: CENET reports that 40–60% of test queries involve such interactions. Because the decay functions are exponentials, the model's memory can be updated incrementally as each new event streams in. Its claims can also be tested ahead of time: it should help exactly on relations whose gaps between repeat events are spread over several timescales.

Core Claim

The claim is narrow and falsifiable. On temporal knowledge graph forecasting, a learned per-relation mixture of exponential decay kernels over elapsed time, plus a static band, should beat a single decay shape. The gain should concentrate on relations whose training-set gaps between related events are multimodal on a log scale. If a single learned decay rate per relation matches the mixture, the multi-scale part of the idea is wrong, and the experiments are designed to show that cleanly.

Each direction plays a different role. Relative temporal encoding is the representation. Multi-scale modeling, borrowed from recommendation, is the mechanism. TKG extrapolation is the application regime. Temporal knowledge representation contributes the static-band idea, through the episodic-to-semantic marginalization of Daxberger et al..

Research Program

Consider a query: subject $s$, relation $r$, unknown object, at time $t_q$. The model scores each candidate object $o$ as

$$ f(o) = \sum_{k=1}^{K} w_{r,k} \sum_{e \in H(s,o)} a_{r,r_e}\, e^{-(t_q - t_e)/\tau_k} \;+\; w_{r,\infty}\, g(s,r,o). $$

The terms are as follows:

  • $H(s,o)$ is the set of historical facts linking $s$ and $o$. These include one-hop facts under any relation $r_e$ and, in a second stage, two-hop rule paths in the style of INFER.
  • $t_e$ is the timestamp of fact $e$.
  • $a_{r,r_e}$ is a learned score for how strongly a past relation $r_e$ predicts $r$.
  • $\tau_1 < \dots < \tau_K$ is a fixed, log-spaced bank of timescales, with $K$ around 8.
  • $w_{r,k} \ge 0$ are the per-relation spectrum weights.
  • $g$ is a static, time-free scoring function in the style of ComplEx.

The design ties to each direction as follows:

  • Relative time. Only elapsed time $t_q - t_e$ enters the score, so unseen future dates pose no problem.
  • Static band. The $w_{r,\infty}$ term is the $\tau \to \infty$ limit of the decay bank. It scores interactions that have never occurred, without a separate classifier such as CENET's.
  • Streaming. Exponentials support a classic recursive update: decay each accumulator, then add the new event. The model therefore keeps $K$ running sums per entity pair and relation, and never stores raw timestamps. This matters for the lifelong streaming setting of Chen et al..

We will build three things. The first is a standalone forecaster defined by the equation above. The second is a drop-in replacement of INFER's square-root validity decay and of TimeTraveler's elapsed-time function with the spectrum. Finding (b) tests whether the idea transfers as an ingredient rather than only as a model. The third is an analysis tool. Before training, it fits a log-space mixture to each relation pair's empirical inter-event gaps and counts the number of modes. That count becomes the pre-registered predictor of where gains should appear.

The closest prior work handles scale differently in each case:

  • DiMNet and MSEN split representations into fast and slow parts. They do so over discrete snapshot indices with heavy graph encoders, not over continuous elapsed time.
  • TeRDy splits relation trajectories by Fourier frequency. It is transductive over a closed time grid and cannot forecast.
  • TPAR encodes relative time with sinusoids plus a linear trend. That form can express periodicity and monotone trend, but not "relevant within days and again within years, then fading."
  • The recommendation work on HierTCN, Time-LSTM and irregular intervals for LLM recommenders motivates the premise that short and long gaps carry different meaning. All of it uses hand-set short/long cutoffs on user-item sequences. We learn the cutoffs, per relation, on multi-relational graphs.

Baselines

The naive reference point is a recency-weighted copy baseline. It predicts the object that most recently co-occurred with the query subject and relation, plus frequency counts. Any temporal model that does not clearly beat it on repeated-fact queries is learning nothing about time.

The strongest existing extrapolation methods in the file are TPAR and MSEN.

  • ICEWS14: TPAR reports 46.89% MRR (TPAR). MSEN reports 46.68% MRR and 68.96% Hits@10 (MSEN). INFER reports 44.46% MRR (INFER).
  • ICEWS18: MSEN reports 34.05% MRR.
  • GDELT: CSI reports 25.6% MRR (CSI).
  • DiMNet: its reported gains are relative improvements only, for example 22.7% MRR on ICEWS05-15. No absolute DiMNet number appears on Lacuna, so it must be re-run.
  • TeRDy: its 0.648 MRR on ICEWS14 is a completion (interpolation) score and is not comparable.

Protocols differ across these papers, especially in how already-true answers are filtered out at test time. We will therefore re-run TPAR, MSEN, DiMNet, INFER, CSI and CENET from released code. Every method gets the same chronological splits, the same time-aware filtered metric, the same GPU budget, and an equal hyperparameter search of 30 trials each on validation MRR. Beating the field means a higher MRR than the best re-run baseline on at least three of the following datasets, with seed-variance intervals that do not overlap: ICEWS14, ICEWS18, ICEWS05-15, GDELT and one of WIKI or YAGO. Being competitive at much lower compute would be a secondary win.

Two ablations isolate the key ingredient. In the first, $K=1$ with a learned rate per relation, which tests single-scale versus multi-scale. In the second, the bank is kept but the static band is removed, which tests the unification with first-time queries. A third ablation fixes INFER's square-root decay inside our scorer.

Why not just…?

Why not just learn one decay rate per relation? This is the most serious threat. It is exactly the $K=1$ ablation. Our pre-registered prediction is specific: the spectrum should win on relations whose gap distributions have two or more modes and tie on unimodal ones. If the mixture shows no advantage even on multimodal relations, we will report that multi-scale structure does not pay off in these benchmarks. The contribution would then shrink to the static-band unification and the streaming update.

Why not just bin log elapsed time and learn a weight per bin? A per-relation-pair histogram over log-time bins can approximate any decay shape. On fixed benchmarks it may match us, and that is a real possibility. It has two concrete weaknesses. It must store timestamps, or per-bin counts that cannot be decayed exactly. And it has no information for gaps longer than any seen during training, which is the normal situation in lifelong forecasting. We test this directly. We train on the early years of ICEWS05-15 and evaluate on queries whose supporting evidence lies further back than the longest training gap, and we also run the ICEWS lifelong streaming splits. If histograms match there too, the exponential parameterization offers no benefit beyond memory cost.

Why not just use TPAR's sinusoid-plus-linear time encoding? We will swap TPAR's encoding into our scorer, holding everything else fixed. Sinusoids never decay, and a linear term cannot be bimodal. We therefore expect TPAR's encoding to lose mainly on ICEWS05-15, where both short bursts and multi-year memory exist. It may win on strictly periodic relations. If so, we would add periodic kernels to the bank rather than argue against them.

Why not just use CENET's two-score gate for first-time facts? CENET counts history over a fixed window and adds a contrastive classifier that switches between a historical score and a non-historical score. Our static band is the continuous limit of the same decay bank. Comparing results on CENET's split into repeated and first-time queries shows whether the extra classifier buys anything.

Proposed Experiments

The decisive check is the pre-registered correlation. For each relation, we compute the number of log-gap modes on training data before any model is fitted. We then measure the per-relation MRR gain of the full model over $K=1$. A positive rank correlation across relations, replicated on ICEWS05-15 and GDELT, supports the claim. No correlation refutes it.

Three secondary checks follow:

  • Transfer as an ingredient. We report MRR when the spectrum replaces INFER's decay and TimeTraveler's elapsed-time function, testing whether the idea carries over to other models.
  • Granularity range. We report results across granularity, from 15-minute GDELT to annual WIKI and YAGO. The learned spectra should shift their weight toward longer timescales on the coarser datasets. If they do not, the model is not reading time the way we claim.
  • Streaming cost. We measure per-event update time and memory against DiMNet and MSEN, measured during inference on the full GDELT stream.