Graph neural networks learn about each node by repeatedly averaging it with its neighbors. After many rounds of averaging, every node looks the same. This failure, called over-smoothing, has been studied almost entirely in static graphs, where "many rounds" means "many layers." Many temporal graph models behave differently: they keep a running memory vector per node and update it with a message from the partner node every time an interaction arrives. On a stream with millions of events, a busy node's memory has been mixed with its neighbors thousands of times. We conjecture that such models are therefore very deep networks along the time axis, and that they may suffer over-smoothing as the stream gets longer, even though each individual update is shallow. If so, this would help explain an embarrassing benchmark fact. On the Temporal Graph Benchmark, a plain moving average beats the memory-based TGN model at node affinity prediction.
This note proposes to test that conjecture directly, and then to fix what it finds with a gate that slows a node's memory update when the update would erase the node's distinctness from its neighbors. The central technical point is that, in a changing graph, a node's measured distinctness moves for two reasons. The model may be smoothing features, or edges may simply be appearing and disappearing. Existing energy-preserving gates cannot tell these apart. Our gate responds only to the part of the change caused by the model's own update, and it measures collapse with a scale-free quantity rather than raw energy.
Core Claim
We make two linked claims. The first is diagnostic: in memory-based temporal graph networks, the effective rank of the node-memory matrix decays as a function of how many events the stream has processed, and this decay tracks the drop in prediction quality on later test windows. Effective rank counts how many independent directions the representations actually occupy. The second is corrective: a gradient gate is computed from only the feature-driven part of the change in local smoothness, with the part caused by topology excluded. Such a gate will stop the decay and recover accuracy, where a naive gate borrowed from static graphs will over-freeze nodes during bursts of new edges.
The Dirichlet energy is a standard measure of how much node features differ across edges. It is the sum over edges of squared feature differences, $\mathcal{E}(X, L) = \tfrac12 \mathrm{Tr}(X^\top L X)$. Here $X$ is the node-feature (here, memory) matrix and $L$ is the graph Laplacian, the matrix that encodes which nodes are connected. Between event $k$ and event $k+1$, both the memory $X_k$ and the graph $L_k$ change, and the energy change splits exactly into two terms:
$$ \Delta\mathcal{E}_k = \underbrace{\mathcal{E}(X_{k+1}, L_{k+1}) - \mathcal{E}(X_k, L_{k+1})}_{\text{caused by the model's update}} + \underbrace{\mathcal{E}(X_k, L_{k+1}) - \mathcal{E}(X_k, L_k)}_{\text{caused by edges arriving or leaving}}. $$
Only the first term reflects smoothing. We compute a node-local version of the first term, restricted to the node's current neighborhood. We divide it by the node's squared memory norm so that rescaling the features cannot hide collapse. The result drives the gate. This is the continuous-time "Gap 1" in the file made operational. The chain-rule decomposition shows that static preservation proofs break once $L$ moves, and our discrete split is the version a streaming model can actually compute.
Relation to Prior Work
The mechanism comes from static-graph work on energy preservation. Gradient Gating multiplies each residual update by a gate derived from the local graph gradient, so that nodes stop updating once they match their neighbors. It keeps Dirichlet energy nonzero through 1,000 layers on a fixed graph. We reuse the gate structure, but its gradient is computed against a fixed topology. On a stream, inserting a dense burst of edges changes the gradient with no smoothing having occurred, and the gate would react to that as if it were smoothing. Our decomposition removes that confound.
ACMP adds repulsive forces between dissimilar nodes and proves a positive energy lower bound. Anti-Symmetric DGN makes layer updates non-dissipative through skew-symmetric weights. Both assume a static adjacency and serve as alternative in-memory update rules to compare against, not as the mechanism we build.
A critique constrains the design. Are we measuring oversmoothing correctly? shows that Dirichlet energy can stay high while features collapse onto one direction, especially under normalization. It also shows that effective rank correlates with accuracy at an average Pearson 0.91 across static benchmarks. We therefore use normalized energy inside the gate and effective rank as the primary diagnostic, and we treat raw energy only as a secondary readout. Because rank alone is not sufficient (random noise has full rank), the diagnostic is always paired with task accuracy.
The application regime comes from temporal graph learning. A Primer on Temporal Graph Learning describes memory-based models such as TGN that update per-node state on each event. Temporal Graph Benchmark supplies the evaluation protocol and the motivating anomaly that heuristics beat TGN. Among existing dynamic-graph work that touches smoothing, TRicci uses temporal curvature to prune edges before a standard model runs, and it never measures representation collapse. FreeDyG filters the frequency content of each node pair's interaction history over time, not the spatial smoothness of node states. Coden updates structural embeddings with fixed personalized PageRank and triggers updates by a structural-change threshold that ignores feature diversity. None of the three treats stream length as network depth or monitors collapse online. We are not aware of any work in the file that measures rank of temporal memory over stream time.
Research Program
The first stage is purely measurement. We train TGN on TGB datasets and log the effective rank, numerical rank, and normalized Dirichlet energy of the memory matrix at regular event intervals. The energy is computed over the edges active in a recent window. We also record test accuracy in sliding windows. We then control effective depth separately from calendar time. We truncate the training history to different lengths, and on held-out windows we stratify nodes by how many memory updates they have received. The hypothesis predicts that high-activity nodes lose distinctness faster and are predicted worse, after controlling for degree. If effective rank does not fall with stream length, or falls without any link to accuracy, the corrective stage is not justified, and we report the negative result.
The second stage builds the gate. At each event touching node $i$, the standard memory update is replaced by a convex combination of the old memory and the proposed update. The mixing weight $\tau_i$ is a learned, channel-wise function of the feature-driven local smoothness change defined above, normalized by the memory norm. Computing it requires the node's current neighbor memories, which TGN's embedding module already samples, so the added cost is one extra neighbor-difference computation per event. The gate is placed only on memory updates, not on the embedding layers, so that any effect can be attributed to time-axis smoothing.
The third stage is a controlled synthetic stream that isolates the drift correction. The stream is generated from a stochastic block model with persistent community labels and features correlated with community. It includes periodic dense bursts of intra-community edges and occasional edge deletions. The task is node community prediction over time. A naive gate should freeze nodes during bursts; a drift-corrected gate should not. Community structure is known, so collapse can be measured against ground truth.
Baselines
The naive reference points are the non-parametric heuristics from the Temporal Graph Benchmark: moving average for node affinity, and EdgeBank-style memory for link prediction. They are run exactly as in that benchmark's protocol. The strongest reported result here is on tgbn-reddit, where moving average reaches NDCG@10 of 0.559 against TGN's 0.315 (Temporal Graph Benchmark). Beating TGN is not enough. The proposal counts as a success on node affinity only if gated TGN closes most of the gap to moving average on the TGB node affinity datasets (tgbn-trade, tgbn-genre, tgbn-reddit, tgbn-token), and ideally surpasses it on at least one.
The main host model is unmodified TGN under the same memory dimension, neighbor sampling, epochs, and hyperparameter search budget. Every variant gets the same number of search trials over learning rate, memory size, and dropout, with three seeds each.
The strongest existing parametric competitor on link prediction is the best reported TGB model. On tgbl-wiki, NAT reaches MRR 0.749 (Temporal Graph Benchmark). On tgbl-review, GraphMixer and TGAT lead. For those two models, no numeric value is reported in the file: no reported result found on Lacuna for GraphMixer or TGAT on tgbl-review. We rerun both under our budget rather than quoting numbers. We do not expect a memory-gating change to top NAT on tgbl-wiki. The link-prediction claim is narrower: gated TGN improves over TGN, and the improvement is concentrated in late test windows and high-activity nodes. For FreeDyG, whose file entry reports only ranks and no numbers, we rerun it on the continuous-time link prediction datasets it uses (Wiki, Reddit, MOOC, LastFM).
The ablation that isolates the key ingredient is naive gradient gating on memory, which uses the raw local gradient against the current neighborhood exactly as in Gradient Gating. A second ablation gates on unnormalized energy instead of normalized energy, which tests the scale critique of Are we measuring oversmoothing correctly?. Two further comparisons replace the memory update with a norm-preserving alternative. One uses pairwise-variance normalization in the style of PairNorm. The other uses a skew-symmetric update in the style of Anti-Symmetric DGN.
Why not just…?
Why not just use the moving-average heuristic, since it already wins? It wins on stable streams because it copies recent history. That same benchmark reports heuristics degrading on high-surprise data such as tgbl-review, and FreeDyG reports density-based heuristics failing on hard negatives. Moving average is still a serious threat on node affinity, which is why it is the primary bar rather than a strawman. If gated TGN does not narrow that gap, the explanation for TGN's weakness lies elsewhere, and the diagnostic stage will show that.
Why not just decay or periodically reset node memory toward its initial value? This is the strongest alternative. Pulling memory back toward a node-specific anchor also counteracts homogenization, and it costs nothing. We include a learned time-decay toward each node's initial embedding, and a fixed reset period, as baselines with the same tuning budget. Our argument is that decay is blind to whether smoothing is happening: it erases useful accumulated history from nodes that are not collapsing. The prediction is therefore specific. On the synthetic stream and on high-activity TGB nodes, decay should trade recent-context accuracy for diversity, while the gate should keep both. If decay matches the gate everywhere, we will report that the simpler fix suffices and that the contribution reduces to the diagnosis.
Why not just apply Gradient Gating unchanged to memory updates? Because edges arriving change the local gradient without any smoothing. The prediction is that a naive gate freezes nodes during edge bursts and under-updates exactly when new information arrives. The synthetic burst stream tests this directly: gate activity during bursts and post-burst accuracy should separate the two variants. If they do not separate, the drift correction is unnecessary, and we will say so.
Why not just add LayerNorm or PairNorm to the memory? Are we measuring oversmoothing correctly? shows that normalization can keep energy high while rank collapses. The PairNorm-style baseline therefore tests whether rescaling alone halts the rank decay we measure. We expect it to hide the energy symptom but not the rank one.
Decisive Checks
Three outcomes settle the proposal. First, the diagnostic: effective rank of TGN memory must fall with stream length. Its correlation with windowed accuracy must be substantial and must survive stratification by node degree. If it does not, the corrective stage stops. Second, the drift-correction test: on the synthetic burst stream, the drift-corrected gate must beat both the naive gate and memory decay in post-burst community accuracy while holding effective rank flat. Third, the benchmark test: gated TGN must close most of the NDCG@10 gap to moving average on the tgbn datasets under the matched budget. On tgbl-wiki and tgbl-review, it must improve on TGN mainly in late windows and for high-activity nodes, the pattern that the depth-along-time hypothesis predicts and that a generic regularization gain would not produce.
References
- Gradient Gating for Deep Multi-Rate Learning on Graphs
- ACMP: Allen-Cahn Message Passing with Attractive and Repulsive Forces for Graph Neural Networks
- Anti-Symmetric DGN: a stable architecture for Deep Graph Networks
- Are we measuring oversmoothing in graph neural networks correctly?
- A Primer on Temporal Graph Learning
- Temporal Graph Benchmark for Machine Learning on Temporal Graphs
- Edge Sparsification via Temporal Forman-Ricci Curvature for Dynamic Graph Learning
- FreeDyG: Frequency Enhanced Continuous-Time Dynamic Graph Model for Link Prediction
- Coden: Efficient Temporal Graph Neural Networks for Continuous Prediction
- PairNorm: Tackling Oversmoothing in GNNs