HTGN-BTW: Heterogeneous Temporal Graph Network with Bi-Time-Window Training Strategy for Temporal Link Prediction

Chongjian Yue, Lun Du, Qiang Fu, Wendong Bi, Hengyu Liu, Yu Gu, Di Yao

Introduction

Temporal link prediction is a critical task in understanding evolving networks, such as e-commerce platforms or social media, where the goal is to predict future interactions between nodes based on historical data. Real-world networks present several challenges: high heterogeneity in node and link types, a lack of descriptive features, and inconsistent time intervals between training data and prediction targets.

To address these, the HTGN-BTW framework was developed, combining a Heterogeneous Temporal Graph Network (HTGN) with a novel Bi-Time-Window (BTW) training strategy. This approach secured 2nd place in the WSDM Cup 2022 Temporal Link Prediction Challenge.

Methodology

The HTGN Architecture

The HTGN architecture extends the Temporal Graph Network (TGN) framework to handle heterogeneity and missing features. Key components include:

  • Static Node and Relation Embeddings: To compensate for missing node features, random vectors are initialized and learned during training. Unique vectors are also assigned to different link types to capture diverse interaction semantics.
  • Temporal Encoding: Instead of raw timestamps, the model utilizes human-centric time features (e.g., hour of day, day of week) to capture periodic behavioral patterns.
  • X-MLP Decoder: A specialized Multi-Layer Perceptron (MLP) is used for each relation type, allowing the model to calculate link probabilities in parallel while specializing in specific interaction types.

Bi-Time-Window (BTW) Training Strategy

The BTW strategy is designed to simulate real-world inference scenarios where a gap exists between historical data and the prediction window. The training process is split into two sliding windows:

  1. Memory Window: Interactions in this window are used solely to update node "memory" (historical states).
  2. Prediction Window: Interactions here are used to calculate loss and perform backpropagation.
The HTGN-BTW framework architecture illustrating the Bi-Time-Window training strategy and the flow from raw messages to link probability prediction.

This separation prevents the model from "cheating" by over-relying on data immediately preceding the target, making it more robust to the unfixed time intervals found in the competition's test set.

Experimental Setup and Datasets

The framework was evaluated on two distinct datasets from the WSDM Cup 2022:

  • Dataset A: A dynamic event graph with 19,442 nodes and over 27 million links across 248 relation types. Both nodes and links have features.
  • Dataset B: A bipartite user-item graph with over 810,000 nodes and 8 million links across 14 relation types. Only links have features.

Results and Discussion

The HTGN-BTW framework demonstrated strong performance across both datasets, achieving 2nd place overall in the competition.

Hyperparameters and AUC performance for Dataset A and Dataset B.

As shown in the experimental results, the model achieved an AUC of approximately 0.662 on Dataset A and 0.907 on Dataset B. The results highlight the effectiveness of the BTW strategy in handling large temporal gaps and the utility of static embeddings when raw node features are absent. The use of different initial weights ($w_0, w_1$) for the temporal attention mechanism across datasets suggests that the relative importance of historical memory versus static identity varies by network type.

Conclusion

HTGN-BTW provides a robust solution for temporal link prediction in heterogeneous networks. By simulating the prediction gap through the Bi-Time-Window strategy and specializing decoders via the X-MLP approach, the model effectively handles the complexities of real-world dynamic graphs. Future work includes automating window size tuning and exploring the integration of raw features where available.

Summary updated

Abstract

With the development of temporal networks such as E-commerce networks and social networks, the issue of temporal link prediction has attracted increasing attention in recent years. The Temporal Link Prediction task of WSDM Cup 2022 expects a single model that can work well on two kinds of temporal graphs simultaneously, which have quite different characteristics and data properties, to predict whether a link of a given type will occur between two given nodes within a given time span. Our team, named as nothing here, regards this task as a link prediction task in heterogeneous temporal networks and proposes a generic model, i.e., Heterogeneous Temporal Graph Network (HTGN), to solve such temporal link prediction task with the unfixed time intervals and the diverse link types. That is, HTGN can adapt to the heterogeneity of links and the prediction with unfixed time intervals within an arbitrary given time period. To train the model, we design a Bi-Time-Window training strategy (BTW) which has two kinds of mini-batches from two kinds of time windows. As a result, for the final test, we achieved an AUC of 0.662482 on dataset A, an AUC of 0.906923 on dataset B, and won 2nd place with an Average T-scores of 0.628942.
BibTeX
@misc{yue2022htgn,
  author = {Chongjian Yue and Lun Du and Qiang Fu and Wendong Bi and Hengyu Liu and Yu Gu and Di Yao},
  title = {{HTGN-BTW: Heterogeneous Temporal Graph Network with Bi-Time-Window Training Strategy for Temporal Link Prediction}},
  year = {2022},
  eprint = {2202.12713},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2202.12713},
}
Figure