Kernelized Edge Attention: Addressing Semantic Attention Blurring in Temporal Graph Neural Networks

Govind Waghmare, Srini Rohan Gujulla Leel, Nikhil Tumbde, B G Sumedh, Sonia Gupta, Srikanta Bedathur

In the world of dynamic graphs—think of a global credit card network or a social media platform—everything moves at different speeds. Your user profile (a node) might change slowly over years, but your transactions or comments (edges) happen in bursts of milliseconds.

A new paper from researchers at Mastercard and IIT Delhi, titled "Kernelized Edge Attention: Addressing Semantic Attention Blurring in Temporal Graph Neural Networks," argues that current models are "temporally colorblind." By mixing these slow and fast signals together, existing Temporal Graph Neural Networks (TGNNs) suffer from semantic attention blurring, making it hard to distinguish a relevant recent interaction from historical noise.

TL;DR

The authors introduce KEAT (Kernelized Edge Attention), a plug-and-play module that uses continuous-time kernels to modulate edge features. By explicitly scaling edge information based on how much time has passed, KEAT eliminates "attention blurring," leading to up to an 18% improvement in link prediction accuracy and significantly better model interpretability.

What Problem Does This Paper Address?

Most TGNNs use attention mechanisms to decide which neighboring nodes are important. However, they typically compute attention scores by summing or concatenating node features (which evolve slowly) and edge features (which are transient and timestamped).

The authors identify a fundamental mismatch here:

  1. Nodes represent entities with long-term states (e.g., a cardholder's creditworthiness).
  2. Edges represent specific, high-frequency events (e.g., a single purchase).

When a model "entangles" these signals, the attention weights often become flat or "blurred." The model can't tell if a high attention score is due to a stable node property or a critically important recent edge. This leads to poor temporal precision—the model knows who is involved but struggles to prioritize when it happened.

Methods

The core innovation is KEAT (Kernelized Edge Attention for Temporal Graphs). Instead of just adding time as a feature, KEAT uses a family of kernels to modulate the edge representations before they enter the attention calculation.

The Secret Sauce: Kernel Modulation

KEAT modifies the standard attention score formula. While the node features remain untouched to preserve structural context, the edge features are scaled by a kernel function $\psi(\Delta t)$:

  • Laplacian Kernel: $e^{-\Delta t/\sigma}$ (Prioritizes recency with a steady decay).
  • RBF (Gaussian) Kernel: $e^{-\Delta t^2/\sigma^2}$ (Provides a smoother, symmetric decay).
  • Learnable MLP: A small neural network that learns the optimal decay pattern directly from the data.

By scaling the edge features this way, the model is forced to "pay more attention" to edges that are temporally relevant. This design is architecture-agnostic, meaning it can be dropped into existing popular frameworks like TGN (Message-Passing style) or DyGFormer (Transformer style) with minimal code changes.

Datasets

The researchers evaluated KEAT across 13 diverse datasets to prove its generality:

  • TGB (Temporal Graph Benchmark): tgbl-wiki, tgbl-review, tgbl-coin, tgbl-comment, and tgbl-flight.
  • JODIE Suite: jodie-reddit, jodie-wiki, jodie-mooc, and jodie-lastfm.
  • Node Classification: tgbn-trade, tgbn-genre, tgbn-reddit, tgbn-token, and the financial dataset DGraphFin.

Key Insights and Ideas

  • Higher-Order Moment Suppression: Theoretically, the authors prove that using these kernels suppresses the "noise" from higher-order moments of the time distribution. This makes the model much more robust to temporal drift—when the timing patterns in your test data look different from your training data.
  • Variance Reduction: The paper demonstrates that kernel modulation reduces the variance of attention logits. This leads to more stable training and more "confident" predictions.
  • Time Encodings Aren't Enough: A major "aha" moment is the realization that simply having time encodings as input features isn't enough. If the attention mechanism doesn't use time to scale the influence of edges, the temporal signal gets lost in the noise of the node embeddings.
  • Interpretability: KEAT produces "sharper" attention heatmaps. In the paper's visualizations, you can clearly see the model shifting its focus as edges age, whereas standard models show a "blurred" uniform focus across all neighbors regardless of time.

Key Results

KEAT consistently outperformed vanilla backbones across almost every metric:

  • Link Prediction: Achieved up to 18.8% improvement in MRR (Mean Reciprocal Rank) on the tgbl-review dataset and 7.7% on tgbl-wiki.
  • Node Classification: On the tgbn-trade dataset, KEAT-TGN saw a 6.9% gain in NDCG@10.
  • Financial Fraud: On the DGraphFin dataset, KEAT improved the ROC-AUC, proving its utility in high-stakes fraud detection where timing is everything.
  • Efficiency: Because the kernels operate on scalar time gaps, KEAT adds negligible computational overhead compared to the base models.

Limitations and Future Work

The authors note that KEAT's effectiveness can dip in specific scenarios:

  • Extreme Sparsity: If a node has very few interactions spread far apart in time, the kernel might downweight the only available information too aggressively.
  • Clustered Activity: In cases where all interactions happen in a very tight temporal window, the kernel modulation might not provide enough contrast to distinguish between them.

Future work may involve exploring adaptive kernels that can change their "width" ($\sigma$) dynamically for different parts of the graph, allowing the model to handle both high-frequency bursts and long-term dependencies simultaneously.

Summary updated

Abstract

Temporal Graph Neural Networks (TGNNs) aim to capture the evolving structure and timing of interactions in dynamic graphs. Although many models incorporate time through encodings or architectural design, they often compute attention over entangled node and edge representations, failing to reflect their distinct temporal behaviors. Node embeddings evolve slowly as they aggregate long-term structural context, while edge features reflect transient, timestamped interactions (e.g. messages, trades, or transactions). This mismatch results in semantic attention blurring, where attention weights cannot distinguish between slowly drifting node states and rapidly changing, information-rich edge interactions. As a result, models struggle to capture fine-grained temporal dependencies and provide limited transparency into how temporal relevance is computed. This paper introduces KEAT (Kernelized Edge Attention for Temporal Graphs), a novel attention formulation that modulates edge features using a family of continuous-time kernels, including Laplacian, RBF, and learnable MLP variant. KEAT preserves the distinct roles of nodes and edges, and integrates seamlessly with both Transformer-style (e.g., DyGFormer) and message-passing (e.g., TGN) architectures. It achieves up to 18% MRR improvement over the recent DyGFormer and 7% over TGN on link prediction tasks, enabling more accurate, interpretable and temporally aware message passing in TGNNs.
BibTeX
@misc{waghmare2026kernelized,
  author = {Govind Waghmare and Srini Rohan Gujulla Leel and Nikhil Tumbde and B G Sumedh and Sonia Gupta and Srikanta Bedathur},
  title = {{Kernelized Edge Attention: Addressing Semantic Attention Blurring in Temporal Graph Neural Networks}},
  year = {2026},
  eprint = {2602.00596},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.00596},
}
Figure