TL;DR
The paper introduces DyGFM, the first multi-domain dynamic Graph Foundation Model designed to train across diverse, time-evolving graph datasets. By decoupling structural semantics from temporal dynamics and using a divergence-aware routing mechanism, DyGFM successfully mitigates "negative transfer" and achieves state-of-the-art performance on downstream tasks with exceptional training efficiency.
What Problem Does This Paper Address?
Graph Foundation Models (GFMs) have recently emerged as a powerful way to learn transferable representations across diverse static graphs. However, extending this success to dynamic graphs—where nodes and edges evolve continuously over time—presents a massive challenge.
Different dynamic graph domains operate on completely incompatible time scales. For example, user interactions on an e-commerce platform happen at a millisecond-level pace, whereas citation networks evolve over years. Directly fusing these heterogeneous data streams during pre-training distorts the temporal semantics (Challenge I). Furthermore, when adapting a pre-trained model to a new target domain, the model often suffers from negative transfer—accidentally applying irrelevant temporal or semantic patterns from a source domain, which degrades downstream performance (Challenge II).
Methods
To tackle these challenges, the authors propose DyGFM, which relies on three core architectural components:
- Dual-Branch Pre-training (Semantic-Temporal Decoupling): Instead of forcing structure and time into a single representation, DyGFM splits them up.
- The Semantic Branch strips away timestamps to focus purely on structural features, utilizing an Information Bottleneck (IB) objective to compress the data and filter out domain-specific noise.
- The Temporal Branch uses domain-specific timers to handle relative time encoding alongside a shared backbone (TGAT) and lightweight adapters to capture unique temporal rhythms.
- Divergence-Aware Expert Routing: To prevent negative transfer during adaptation, DyGFM treats pre-trained source domains as "experts" in a Mixture-of-Experts (MoE) framework. It mathematically measures the semantic divergence (via KL divergence of Gaussian distributions) and temporal divergence (via Euclidean distance) between the target domain and source domains, routing data to the most relevant experts.
- Divergence-Conditioned Prompting: Rather than fine-tuning the entire model backbone—which is computationally heavy and risks destroying pre-trained knowledge—DyGFM keeps the backbone frozen. It introduces a prompt generator that injects lightweight, learnable graph prompts conditioned on the calculated domain divergences to guide downstream adaptation.
Datasets
The model was evaluated on four widely used, real-world continuous-time dynamic graph benchmarks:
- Wikipedia: A user-page editing network tracking user ban states.
- Reddit: A user-post interaction network on subreddits tracking user ban states.
- MOOC: A student-course activity network predicting student dropouts.
- Genre: A large-scale user-genre music listening network predicting user preferences.
Key Insights and Ideas
- Time is Relative: Trying to enforce a single global clock across multiple distinct domains destroys a foundation model's utility. Decoupling time from structure allows the model to learn universal structural properties while letting individual domains maintain their native temporal granularities.
- Statistical Divergence as a Router: Explicitly calculating how "far away" a target domain is from source domains prevents the model from getting confused by irrelevant source knowledge during few-shot adaptation.
- Prompting Isn't Just for LLMs: Injecting lightweight tokens into frozen graph encoders provides a massive boost in training efficiency and prevents the catastrophic forgetting of graph structures.
Key Results
DyGFM was tested against 12 state-of-the-art baselines under two main settings: All-Seen-Domain-Adaptation (ASDA) and Leave-One-Domain-Out (LODO).
- Superior Accuracy: In the ASDA setting, DyGFM consistently achieved the highest AUC-ROC scores, outperforming the closest dynamic prompt baseline (DyGPrompt) by up to 5.0% on MOOC node classification. In the LODO setting (testing on completely unseen domains), it achieved an 8.0% improvement on MOOC node classification and a 7.1% improvement on Reddit link prediction.
- High Efficiency: DyGFM demonstrates remarkable training efficiency. As shown in the efficiency analysis, it converges significantly faster than its competitors—stabilizing within just 56 epochs compared to over 200 epochs for other dynamic prompting baselines—while maintaining a highly optimized GPU memory footprint.
Limitations and Future Work
Not explicitly discussed in the paper.
Summary updated