Spike Arrival Times as Hyperbolic Geometry for Temporal Knowledge Graphs

Hyperbolic embeddings work well for knowledge graphs because they behave like continuous trees. Distances grow so fast toward the edge of the space that many branches of a hierarchy fit without crowding. That benefit has a price. Every layer needs hyperbolic trigonometric maps, and those maps become numerically unstable near the boundary, which is the opposite of what low-power neuromorphic chips can run. Spiking networks have a different and underused property. When a spike travels through a network whose edges carry delays, the time it first reaches a node equals a shortest-path distance. Shortest-path distances on a sparse relational graph are close to tree metrics, which are exactly the metrics hyperbolic space approximates. We propose to use first-spike latency as the embedding geometry for temporal knowledge graphs. Temporal knowledge graphs are databases of timestamped facts such as (country A, signs treaty with, country B, 2014). In our model, a query (subject, relation, ?, time) launches a spike wave from the subject, relation and time adjust the synaptic delays, and candidate objects are ranked by how early the spike reaches them. Hierarchy is handled with no exp/log maps, only additions and comparisons.

The obstacle is training. A strict first-spike network passes gradient only through the single winning path. That is the spiking form of vanishing gradients, and it is why straight-through estimators on discrete delays become unstable on deeper graphs. Our central technical observation is that the smooth relaxation of "first arrival" is a log-sum-exp with a temperature. One temperature knob then sets both how sharp the surrogate gradient is and how tree-like (how strongly curved) the learned metric is. The two problems the literature treats separately, surrogate-gradient degradation in spiking networks and curvature selection in hyperbolic models, become a single annealing schedule we can study.

Core Claim

Let $\tau_v$ be the arrival time of the query spike at entity $v$, and let $d_{uv}(r,t) \ge 0$ be the learned delay on the edge from $u$ to $v$ when answering relation $r$ at time $t$. The relaxed arrival update is

$$\tau_v^{(k)} = -\varepsilon \log\Big( e^{-\tau_v^{(k-1)}/\varepsilon} + \sum_{u \in \mathcal{N}(v)} e^{-(\tau_u^{(k-1)} + d_{uv}(r,t))/\varepsilon} \Big),$$

where $k$ indexes propagation steps, $\mathcal{N}(v)$ is the set of neighbors with an active edge at time $t$, and $\varepsilon > 0$ is the temperature. As $\varepsilon \to 0$ this becomes the exact min-plus Bellman–Ford step, and the network is a pure first-spike delay network. At finite $\varepsilon$, the gradient with respect to each incoming path is that path's softmax weight, so losing paths still receive signal.

Two structural facts follow. First, the self term $e^{-\tau_v^{(k-1)}/\varepsilon}$ is an identity shortcut in min-plus form. It plays the same role as the post-activation addition in SEW ResNets, which preserved identity gradients through 152 spiking layers, and it keeps gradients from vanishing as the number of propagation steps $k$ grows. Second, this log-sum-exp relaxation is the ultradiscretization link that UltraLIF used to derive exactly differentiable spiking dynamics in max-plus algebra. We reuse that algebra, but for metric computation over a graph rather than for single-neuron dynamics.

The geometric claim connects to hyperbolic models. Hyperbolic space rescaled toward large curvature converges to a tree, a standard fact in metric geometry. Lowering $\varepsilon$ therefore pushes the learned latency metric toward a tree metric, in the way that raising curvature makes a hyperbolic embedding more tree-like. We predict three things. Latency embeddings should match hyperbolic temporal models on hierarchical datasets. Their measured tree-likeness should follow the $\varepsilon$ schedule. And at inference the model can run at $\varepsilon = 0$, as strict binary first-spike events.

A finite-temperature version can also be realized on spiking hardware. Each entity is a small population of neurons, and each neuron receives independently jittered delays drawn from a Gumbel distribution scaled by $\varepsilon$. The population's earliest spike then estimates the log-sum-exp up to a known constant. The same mechanism also reuses the population coding of GRSNN.

Research Program

The method builds directly on GRSNN. GRSNN encodes knowledge-graph relations as learnable synaptic delays, treats link prediction as spike propagation (a differentiable Dijkstra), and trains with surrogate gradients plus a straight-through estimator on the delays. It reports results competitive with NBFNet at a theoretical 20× energy saving, but only on static graphs and only in what amounts to Euclidean latency space. We change three things. We add time, we replace the straight-through estimator with the annealed relaxation above, and we treat the latency metric as a hyperbolic object that we analyze and test against hyperbolic baselines.

Time enters through the delays. Each delay is a nonnegative function of a per-relation base delay plus a query-dependent modulation of a continuous time encoding. This follows GRSNN's neuromodulatory query conditioning, except that the modulation now depends on $t$, so the model can handle timestamps and intervals it never saw. Facts with intervals open edges only during their interval. To keep gradients stable across continuous-valued delays we borrow the Gaussian delay kernels of Delay-DSGN, together with its derived constraints on kernel spread and size. Delay-DSGN established those constraints for node classification on evolving graphs, which have one relation type and no quadruple queries. We test whether the same constraints are enough in the multi-relational setting.

The analysis has three parts. The first is a gradient-flow study across the number of propagation steps, at several fixed temperatures. We measure gradient norms per hop and the fraction of edges that receive nonzero gradient, and compare against the straight-through estimator. The second is a geometry study. We compute Gromov δ-hyperbolicity (a standard measure of how tree-like a metric is) of the learned latency metric. We also compute distortion on the type and location hierarchies of YAGO11k, as functions of $\varepsilon$. We compare these numbers with those of the curvature parameters learned by hyperbolic baselines. The third is an energy and operations accounting. We count synaptic additions and comparisons per query and set them against the multiply-accumulates and transcendental-function calls of the hyperbolic models.

Baselines

The naive reference point is a Euclidean temporal embedding with no hierarchy-aware geometry. We use HyTE, from HyTE, which projects entities onto a hyperplane for each time interval. We pair it with the SANe numbers that HyGNet reports, MRR 0.250 on YAGO11k and 0.432 on Wikidata12k. Any latency model that fails to beat these is not worth pursuing.

The strongest existing method is HyGNet, from Hyperbolic Graph Neural Network for Temporal Knowledge Graph Completion. HyGNet does time-gated message passing and convolution in a Poincaré ball, with a loss that keeps embeddings consistent across adjacent timestamps. It is the strongest competitor because it is hyperbolic, temporal, and evaluated on exactly the hierarchical interval datasets where our geometric claim should matter most. It reports MRR 0.276 on YAGO11k and 0.450 on Wikidata12k, and a 6.8% MRR gain over SANe on unseen timestamps. We will rerun HyGNet with its released hyperparameters under our evaluation protocol: the same filtered ranking, the same splits, and the same number of negative samples. Both HyGNet and our model get the same hyperparameter-search budget, counted in GPU-hours. A win has two forms. The strong form is matching or exceeding HyGNet's reproduced MRR. The weaker but still decisive form is staying within 0.015 MRR of it while using at least ten times fewer estimated arithmetic operations per query, with no transcendental operations at inference.

The second hyperbolic baseline is HERCULES and its non-temporal parent ATTH, from Hyperbolic Temporal Knowledge Graph Embeddings with Relational and Time Curvatures. That paper found that ATTH, with no time information but 200 to 500 negative samples, matches temporal models on the ICEWS datasets, and that corrupting timestamps barely changes performance. We run ATTH at 500 negatives and give our model the same negatives. On ICEWS14 and ICEWS05-15 we found no reported MRR result on Lacuna for either model, so those comparisons rely entirely on our reproduction. For forecasting future snapshots we compare against HyperVC, from Bending the Future. HyperVC reports up to an 8.27% Hits@1 gain over RE-Net on WIKI, but the file gives no absolute value, so we reproduce it.

The spiking baseline is GRSNN extended naively to time, which we call GRSNN-T. Time is appended as an extra relation-like conditioning signal, and training uses GRSNN's original straight-through estimator. We found no reported MRR result on Lacuna for GRSNN on FB15k-237 or WN18RR. We therefore first reproduce it on FB15k-237 to validate our implementation before extending it.

The key ablation keeps our architecture fixed and varies only the training relaxation. The three conditions are a straight-through estimator at $\varepsilon = 0$, a fixed $\varepsilon$, and an annealed $\varepsilon$. A second ablation removes the self term to isolate the identity shortcut.

Why not just…?

Why not just run GRSNN with timestamps added? This is the most serious threat. If GRSNN-T trained with the straight-through estimator matches our annealed model, then the gradient contribution is nil, and the proposal reduces to "GRSNN on temporal data." The temperature ablation settles this directly. We expect the gap to grow with the number of propagation steps, because the straight-through estimator updates only the winning path. If the gap does not appear, we will report that and narrow our claim to the geometric analysis.

Why not just train HyGNet and convert it to spikes afterward? Conversion could binarize the activations, but HyGNet maps into and out of the tangent space with log and exp at every convolutional layer. Those $\tanh$ and $\operatorname{arcosh}$ evaluations stay dense and transcendental after conversion, and the boundary instabilities of the Poincaré ball remain. We will include a converted HyGNet in the operations accounting. If it reaches competitive MRR at a comparable operation count, that would weaken the case for native latency geometry, and we would say so.

Why not just use a well-tuned low-dimensional embedding with many negatives? HERCULES shows that heavy negative sampling lets a time-agnostic hyperbolic model compete on ICEWS. Time may therefore be a weak signal there for every model, ours included. That is why we give equal negatives to all models, and why our primary evidence comes from YAGO11k, Wikidata12k, and the unseen-timestamp split, where HyGNet's results suggest time does carry information. We also run the timestamp-corruption test from HERCULES on our own model. If corrupting timestamps leaves our MRR unchanged, our temporal delays are acting as regularizers and not doing reasoning.

Proposed Experiments

The first experiment is temporal link completion on YAGO11k and Wikidata12k, measured by filtered MRR and Hits@1/3/10, against all the baselines above. The second applies the same completion task to ICEWS14 and ICEWS05-15, adding the timestamp-corruption control. The third tests generalization to unseen timestamps, following HyGNet's protocol, to probe the continuous time encoding. The fourth is forecasting on ICEWS18, WIKI, and YAGO against a reproduced HyperVC. HyperVC reports its largest gains on WIKI and YAGO, the most hierarchical of these benchmarks, and lower hierarchy scores on ICEWS18. We therefore predict that our margin over GRSNN-T will be largest on YAGO and WIKI and smallest on ICEWS18.

Three checks decide the outcome. First, the annealed relaxation must beat the straight-through estimator by a margin that grows with propagation depth. Second, the measured δ-hyperbolicity of the latency metric must fall as $\varepsilon$ decreases. Its correlation with downstream MRR on hierarchical relations must be comparable to the link between curvature and performance that HyperVC reports. Third, running at $\varepsilon = 0$ with pure first-spike inference must lose less than 0.01 MRR relative to finite-temperature inference, while keeping the operation savings. If all three hold, spike timing is a hardware-native substitute for hyperbolic coordinates. If the first fails but the second holds, the geometric finding stands, and the training contribution goes back to GRSNN.