DPCL-Diff: The Temporal Knowledge Graph Reasoning based on Graph Node Diffusion Model with Dual-Domain Periodic Contrastive Learning

Yukun Cao, Lisheng Wang, Luobing HuangarXiv 2024 · DOI · arXiv

Introduction

Temporal Knowledge Graphs (TKGs) represent the evolution of relational data over time, structuring facts as quadruplets $(subject, relation, object, timestamp)$. A primary challenge in TKG research is "extrapolation"—the ability to predict future events based on historical patterns. However, existing models often struggle with two critical issues: the "cold start" problem involving entities with sparse historical data, and the "similarity trap" where periodic events with different outcomes are mapped too closely in traditional geometric spaces.

This paper introduces DPCL-Diff, a novel framework that integrates a Graph Node Diffusion (GNDiff) model with Dual-Domain Periodic Contrastive Learning (DPCL). By combining generative modeling to address data sparsity with hyperbolic geometry to better distinguish complex relational patterns, DPCL-Diff achieves state-of-the-art performance across multiple benchmarks.

Problem Statement

1. The Sparse Trace Problem

In real-world TKGs, a significant portion of events—often up to 40%—involve "new" interactions where the entities have little to no prior history. Traditional reasoning models rely heavily on historical frequency and repetition. When faced with these sparse traces, these models lack the necessary context to make accurate predictions, leading to a sharp decline in performance for novel events.

2. The Similarity Trap

Many events in TKGs are periodic, occurring with the same subject and relation but involving different objects over time. In standard Euclidean space, these entities often end up with nearly identical representations because they share similar contexts. This lack of discriminative power makes it difficult for models to distinguish between distinct but contextually similar entities, a phenomenon referred to as the similarity trap.

Methodology

DPCL-Diff addresses these challenges through a bifurcated architecture that handles generative data augmentation and geometric representation separately.

Graph Node Diffusion (GNDiff)

To solve the sparse trace problem, the authors treat the emergence of new events as a generative process. GNDiff utilizes a diffusion-based approach:

  • Forward Process: The model introduces noise into the event data by systematically "masking" nodes, simulating a state of inactivity or missing information.
  • Reverse Process: The model learns to denoise these states, effectively "hallucinating" or generating plausible historical contexts for entities with sparse data. This process aligns the model with the actual distribution of real-world event occurrences, providing the necessary synthetic history to support reasoning for new events.

Dual-Domain Periodic Contrastive Learning (DPCL)

To resolve the similarity trap, the framework employs a dual-domain strategy that selects the optimal geometric space based on the nature of the event:

  • Periodic Events in Poincaré Space: Recurring events are mapped into hyperbolic (Poincaré) space. Hyperbolic geometry is uniquely suited for hierarchical and clustered data because the space expands exponentially toward the edge of the disk. This provides the "room" necessary to separate entities that appear in similar contexts, allowing for finer distinction between periodic interactions.
  • Non-Periodic Events in Euclidean Space: Standard, non-recurring events are maintained in Euclidean space, where their simpler, linear relationships can be modeled efficiently.

The final prediction is derived from a weighted integration of the probabilities produced by both the GNDiff and DPCL modules, balanced by an equilibrium coefficient $\alpha$.

Experimental Evaluation

Datasets

The model was evaluated on four standard TKG benchmarks: ICEWS18, ICEWS14, WIKI, and YAGO. To specifically test the model's robustness against the cold start problem, the authors also curated "New Events" subsets, focusing exclusively on first-time occurrences within the test sets.

Key Results

DPCL-Diff consistently outperformed 12 baseline models, including state-of-the-art frameworks like CENET and RLGNet. Notable findings include:

  • Significant Performance Gains: On the ICEWS14 dataset, which contains a high volume of new events, DPCL-Diff achieved a 30.37% improvement in Hits@1 over the previous best-performing model.
  • Effectiveness of Diffusion: In head-to-head comparisons on the "New Event" subsets, the GNDiff module significantly outperformed standard contrastive learning methods, validating the use of generative diffusion for sparse data.
  • Ablation Studies: Removing either the diffusion component or the hyperbolic mapping resulted in a measurable performance drop, confirming that both innovations are essential to the model's success.

Discussion and Conclusion

DPCL-Diff demonstrates that the integration of generative diffusion and non-Euclidean geometry provides a powerful solution to the inherent limitations of TKG reasoning. By using diffusion to "fill the gaps" in sparse historical data and hyperbolic space to untangle similar relational patterns, the model sets a new standard for temporal extrapolation.

Future work will focus on optimizing the computational efficiency of the diffusion process to support larger batch sizes and scaling the framework to industrial-sized knowledge graphs containing millions of nodes. Additionally, automating the selection of the equilibrium coefficient $\alpha$ remains a priority to ensure the model can adapt seamlessly to varying dataset characteristics.

Summary updated

BibTeX
@misc{cao2024dpcl,
  author = {Yukun Cao and Lisheng Wang and Luobing Huang},
  title = {{DPCL-Diff: The Temporal Knowledge Graph Reasoning based on Graph Node Diffusion Model with Dual-Domain Periodic Contrastive Learning}},
  year = {2024},
  eprint = {2411.01477},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.01477},
}
Figure