Do Temporal Graph Models Over-Smooth Across Time?

Models that learn from evolving networks, such as streams of transactions, edits or messages, keep a running summary vector for each node and update it every time that node takes part in an event. Each update blends the node's old summary with information from its neighbors. On static graphs, repeated blending of this kind is the classic cause of over-smoothing: as more message-passing layers are stacked, node representations collapse toward a few shared directions and stop telling nodes apart. We propose that the same collapse happens along the time axis. Thousands of event updates act like thousands of layers, so node memories slowly lose dimensionality. If that is true, it would help explain a result that has embarrassed the field: on the Temporal Graph Benchmark, a moving average beats a deep temporal network by a wide margin.

The proposal has two parts. The first is a diagnostic study. We measure the effective rank of the matrix of node memories (the entropy of its singular values, which counts how many independent directions the representations really span) as a function of stream time, and test whether it predicts the drop in accuracy over time. The second is an intervention. We replace the dissipative memory update with a conservative one in which an energy-preserving, rotation-like update keeps rank by construction. A learned damping term is switched on only where the stream signals real structural change. This is where the stability–expressivity trade-off comes in: pure conservation preserves information but also preserves noise, and damping buys stability at the cost of expressivity. Our claim is that the right place to spend that trade-off is decided by observed drift, not by a fixed constant.

Core Claim

We make three testable claims. First, memory-based temporal graph networks show temporal rank collapse: the effective rank of node states decays with the length of the stream. Second, this decay predicts per-window performance loss beyond what the benchmark's "surprise index" (the fraction of test edges never seen in training) already explains. Third, a port-Hamiltonian memory, one that splits each node state into position-like and momentum-like halves, updates them with an energy-conserving step, and adds drift-gated friction, holds rank steady and improves accuracy on long streams without becoming more sensitive to spurious edges.

The mechanism has a simple reason to work. A linear update of the form $h \mapsto \exp(\Delta t\, S)\, h$, where $S$ is skew-symmetric ($S = -S^\top$), is an orthogonal map. Applied as a shared transform to the node-state matrix $X$, it leaves every singular value of $X$ unchanged, so the conservative part of the update cannot reduce rank. Rank can then fall only through the damping term, and damping is the part we control explicitly.

Grounding in Prior Work

The diagnostic tool comes from Are we measuring oversmoothing in graph neural networks correctly?. That paper shows that Dirichlet energy, a measure of how different neighboring nodes' features are, stays deceptively high in trained networks even after accuracy collapses. Effective rank, by contrast, correlates with accuracy at an average Pearson $r = 0.91$. That work is entirely static and spatial. We move the same measurement onto the time axis, and we take its negative result as a constraint: we do not use Dirichlet energy as our signal.

The motivating evidence that something is wrong with deep temporal memories comes from Temporal Graph Benchmark for Machine Learning on Temporal Graphs. On tgbn-reddit, a moving average reaches NDCG@10 of 0.559 against 0.315 for the Temporal Graph Network (TGN), while learned models win on high-surprise streams such as tgbl-review. Coden scales continuous prediction to Papers100M by combining parameter-free Personalized PageRank (PPR) with state-space models and a lazy update rule triggered by accumulated drift. However, PPR is a low-pass operator, and that paper never checks whether representations collapse over long horizons. Coden supplies two things we reuse: the idea of gating computation on drift, and the most scalable baseline. Graph-Time Convolutional Neural Networks proves that perturbation sensitivity grows with the temporal window $T$, which is a warning that simply making temporal models deeper or longer trades away stability. Weisfeiler-Lehman goes Dynamic shows that telling dynamic graphs apart can demand very deep temporal unfolding. Together these two papers frame the dilemma: depth over time is needed for expressivity, yet it threatens stability and, we hypothesize, rank.

The method draws on three non-dissipative designs. Anti-Symmetric DGN uses skew-symmetric weights to keep deep static propagation from vanishing or exploding. Its authors note that it was not designed for over-smoothing and leave dynamic graphs open. Port-Hamiltonian Architectural Bias for Long-Range Propagation adds position/momentum coordinates, learnable damping and symplectic integration across up to 1,000 layers, again on fixed topologies and without rank metrics. Non-Dissipative Graph Propagation for Non-Local Community Detection reports that strict conservation hurts on homophilic graphs, where smoothing is useful. This negative result motivates gated rather than absent damping. Our difference from all three is that the conservative dynamics run over event time, not over layer depth, on a graph whose edges change. Damping is also scheduled by stream drift rather than fixed.

The multi-channel filters of Complete the Missing Half are a simpler anti-smoothing alternative that we test directly. They add a high-pass channel alongside ordinary aggregation and improve heterophilic accuracy by roughly 10%.

Research Program

The first phase is measurement and does not depend on any new architecture. We instrument TGN, GraphMixer, TGAT and Coden on the TGB node-affinity and link-prediction tasks. At regular points in stream time, we record the effective rank and numerical rank ($\|X\|_F^2/\|X\|_2^2$) of the memory matrix of active nodes, alongside the rolling task metric and the windowed surprise index. We then fit a per-window regression of the metric on surprise and on rank, to test whether rank explains variance that surprise does not.

A synthetic control separates two causes of rank decay. In it, we hold the generating process fixed and lengthen the stream. If rank falls, the cause is the update rule. If it stays steady, then real streams lose rank because of data drift, and the update rule is not to blame.

The second phase builds the drift-gated port-Hamiltonian memory. On each event involving node $i$ at time $t$, the state $(q_i, p_i)$ is advanced by one symplectic Euler step:

$$p_i \leftarrow p_i - \Delta t\left[\nabla_{q_i} H(q, p;\, \mathcal{N}_t(i), e_t) + g_i(t)\, D\, p_i\right], \qquad q_i \leftarrow q_i + \Delta t\, \nabla_{p_i} H.$$

Here $H$ is a learned energy function that couples node $i$ to its current neighbors $\mathcal{N}_t(i)$ and the event features $e_t$. The symbol $\Delta t$ is the elapsed time, $D$ is a learned positive semi-definite friction matrix, and $g_i(t) \in [0,1]$ is a gate computed from local drift. We consider two forms for the gate: the change in effective rank of $i$'s neighborhood states since its last update, and an event-novelty score for unseen edges. With the gate closed, the update conserves energy and rank. With it open, friction lets the memory forget stale structure.

The analysis component aims for two bounds. The first limits how far a single spurious event can move a node's state, independent of stream length, in contrast to the growth with $T$ in Graph-Time Convolutional Neural Networks. The second shows that rank loss is bounded by the accumulated gated damping.

Baselines

The naive reference points are the TGB heuristics: Moving Average for node affinity and EdgeBank for link prediction. These are the honest "do nothing" models. On tgbn-reddit, the number to beat is the Moving Average NDCG@10 of 0.559 reported in Temporal Graph Benchmark. Any learned memory that fails to clear it has not shown value on that task.

The strongest learned baselines are TGN, the memory model our diagnosis most directly targets, at 0.315 NDCG@10 on tgbn-reddit per the same paper, and NAT on link prediction, which TGB reports at MRR 0.749 on tgbl-wiki. GraphMixer and TGAT are included because TGB reports that they lead on the high-surprise tgbl-review, although the file gives no number for them there. Coden is the scalable, low-pass competitor. On its DBLP, Tmall, Reddit and Patent streams, no reported accuracy number was found on Lacuna, so we rerun it ourselves.

All models use TGB's official splits and negative sampling, the same memory dimension, and an equal hyperparameter search budget per model (the same number of trials on the validation set, with the same time limit per trial). Results are reported as mean and spread over five seeds.

Three ablations isolate the key ingredient. The first uses pure conservation (gate always closed, A-DGN-style). The second uses fixed damping (gate a constant, PH-DGN-style). The third keeps the gate but replaces the symplectic update with a standard recurrent update, which tests whether conservation or gating carries the gain.

We count the proposal as succeeding if it beats TGN and matches or exceeds the heuristic on tgbn-reddit, beats NAT on tgbl-wiki within seed variance, and leads on tgbl-review, all while holding effective rank flat where TGN's declines.

Why not just…?

Why not just use a moving average, since it already wins? On low-surprise node-affinity tasks it might keep winning, and we will report that plainly. However, TGB shows that persistence heuristics fail when many test edges are new, as on tgbl-review. Our claim is narrower: rank collapse is part of why learned memories lose to heuristics on long streams. The per-window regression tests that claim directly. If rank adds nothing beyond surprise, the motivation for the intervention is weakened.

Why not just add an explicit effective-rank penalty to TGN's loss? This is the most serious threat, because it targets the measured quantity directly. A rank penalty can, however, be satisfied by injecting directions that carry no information, since random noise has maximal rank, as the over-smoothing measurement paper itself warns. It also does nothing to bound sensitivity to spurious edges. We include TGN with a rank penalty, tuned under the same budget, as a baseline. If it matches our model on both accuracy and the perturbation test, we would conclude the architecture is unnecessary and report that outcome.

Why not just reset or periodically re-initialize node memories, or add a high-pass channel? Periodic resets restore rank but throw away long-range history, which is exactly what deep temporal unfolding is meant to retain according to Weisfeiler-Lehman goes Dynamic. A high-pass channel in the style of Complete the Missing Half counters smoothing across neighbors at a single moment, not across repeated updates in time. Both are cheap enough to run as baselines: TGN with reset schedules tuned on validation, and TGN with a filterbank aggregation. If either closes the gap, our gating mechanism is not needed.

Why not just monitor Dirichlet energy? The measurement paper shows that it stays high while accuracy collapses in trained networks. We will report it alongside rank only to confirm that the same failure appears over time.

Decisive Checks

The whole project hinges on the first experiment. If TGN's and Coden's memory effective rank does not decline with stream length on tgbn-reddit, tgbl-wiki and the synthetic fixed-process stream, the premise is false. We would then publish the negative diagnostic and stop the architectural work.

If rank does decline, the second check asks whether rank predicts windowed metric drops after controlling for the surprise index.

The third check is the stability test. We inject random spurious edges into the test stream at rates of 1%, 5% and 10%, and measure both the change in node embeddings and the metric drop. Pure conservation should be the most fragile, fixed damping the most sluggish, and the gated model should be best at both. A failure here would mean the gate does not resolve the trade-off.

Finally, we report throughput against Coden on its larger streams. A model that wins on accuracy but cannot process streams of that size would not answer the question that motivated Coden in the first place.