Abstract
Learning representations for graph-structured data is essential for graph analytical tasks. While remarkable progress has been made on static graphs, researches on temporal graphs are still in its beginning stage. The bottleneck of the temporal graph representation learning approach is the neighborhood aggregation strategy, based on which graph attributes share and gather information explicitly. Existing neighborhood aggregation strategies fail to capture either the short-term features or the long-term features of temporal graph attributes, leading to unsatisfactory model performance and even poor robustness and domain generality of the representation learning method. To address this problem, we propose a Frame-level Timeline Modeling (FTM) method that helps to capture both short-term and long-term features and thus learns more informative representations on temporal graphs. In particular, we present a novel link-based framing technique to preserve the short-term features and then incorporate a timeline aggregator module to capture the intrinsic dynamics of graph evolution as long-term features. Our method can be easily assembled with most temporal GNNs. Extensive experiments on common datasets show that our method brings great improvements to the capability, robustness, and domain generality of backbone methods in downstream tasks. Our code can be found at https://github.com/yeeeqichen/FTM.
1. Introduction
In the real world, graphs are rarely static; social networks, financial transaction logs, and citation indexes evolve constantly. Representing these temporal graphs is challenging because the "neighborhood" of a node changes over time. The primary bottleneck in current temporal graph representation learning is the neighborhood aggregation strategy.
Existing approaches generally fall into two categories, both with significant flaws. Snapshot-based methods slice time into discrete buckets, treating everything within a bucket as a static graph and thereby losing fine-grained "short-term" sequences. Random walk-based methods sample past neighbors to capture temporal order but often fail to find a consistent balance between immediate interactions and broader historical trends. These limitations hinder the model's ability to capture the intrinsic dynamics of node evolution.
2. Methodology
The core innovation of FTM is the introduction of a link-based framing technique coupled with a two-stage aggregation process to capture both short-term and long-term features.
2.1 Link-based Framing
Inspired by signal processing techniques like windowing, FTM employs a Link-based Framing Technique. Instead of slicing time by fixed intervals (e.g., hours or days), FTM slices the graph based on a fixed number of interactions ($k$). For a target node, the model takes the most recent $k$ links to form a "frame" and then shifts back by a "hop length" to create a sequence of successive frames. This ensures every frame has a consistent amount of data, avoiding the "data bias" where active periods overwhelm the model while quiet periods provide no information.
2.2 Two-Stage Aggregation Architecture
FTM processes these frames through a modular two-stage architecture:
- Stage 1: Frame Aggregator: Each frame is processed in parallel using a standard GNN backbone (such as TGAT or TGN). This extracts "short-term" features specific to that small window of interactions.
- Stage 2: Timeline Aggregator: The model aggregates the representations from successive frames. This allows the model to "see" the trajectory of the node's evolution, capturing "long-term" features and intrinsic dynamics.
3. Experimental Evaluation
The authors evaluated FTM across seven diverse real-world datasets, including user-interactions (Reddit, Wikipedia, Mooc), political events (Icews), and trust networks (Bitcoin).
3.1 Performance and Domain Generality
Assembling FTM with backbones like TGN or TGAT consistently outperformed original models. On the Wikipedia dataset, FTM provided an average gain of 2.98% in Average Precision (AP) for inductive tasks. Most strikingly, FTM demonstrated exceptional domain generality. When trained on Reddit and tested on Icews14 (a completely different domain), FTM improved baseline performance by 20.42% AP, suggesting it learns the underlying logic of graph evolution.
3.2 Robustness to Noise
The framing approach acts as a natural filter, creating a "scrubbing" effect where noisy or irregular links that deviate from the broader trend are de-emphasized. In adversarial tests with Gaussian noise added to link features, FTM-enhanced models maintained high accuracy, improving AUC by over 18% at high attack intensities compared to standard models.
4. Conclusion and Future Work
FTM is a powerful, plug-and-play method that significantly enhances temporal GNNs by capturing multi-scale temporal features. While it introduces some computational overhead due to parallel frame processing and requires tuning of frame and hop lengths, its superior performance and robustness make it a valuable tool for temporal graph analysis. Future work may explore more complex sequence models, such as Transformers or LSTMs, for the timeline aggregator stage to further improve performance.
Summary updated