Neural Latent Space Model for Dynamic Networks and Temporal Knowledge Graphs

Tony Gracious, Shubham Gupta, Arun Kanthali, Rui M. Castro, Ambedkar DukkipatiAAAI 2021 · DOI

Abstract

The Neural Latent Space Model (NLSM) is a unified statistical framework designed to model the evolution of both simple (homogeneous) and complex (heterogeneous/Knowledge Graph) networks over time. By combining traditional latent space modeling with deep learning-based variational inference, NLSM achieves state-of-the-art performance in link forecasting while maintaining model interpretability.

Introduction

Most real-world networks—from social circles to international diplomacy—are dynamic. However, many representation learning techniques treat them as static snapshots. Existing methods for dynamic networks generally fall into two silos: those for homogeneous networks (one type of node and edge) and those for Knowledge Graphs (KGs) (multiple types of entities and relations). The challenge lies in creating a model that is unified, temporal, and scalable.

Methodology

The NLSM builds on several key pillars:

  • Latent Space Representation: Each node is assigned a latent vector $z_n^{(t)}$ representing its attributes at time $t$.
  • Interaction Matrices: The model uses interaction matrices $\Theta_k^{(t)}$ to define how specific attributes contribute to edge formation, capturing both homophily and heterophily.
  • Minimalist Dynamics: Node attributes and interaction rules evolve via a Gaussian random walk, preventing overfitting to noise.
  • Neural Variational Inference: The authors parameterize the variational distribution using Gated Recurrent Units (GRUs), which output the parameters for latent variables based on network history.
  • Heterogeneous Extension: For KGs, the model assigns unique interaction matrices to each relation type, allowing the same node embeddings to interact differently depending on the relationship.

Experimental Evaluation

The model was evaluated across diverse dynamic networks, including homogeneous/bipartite datasets (Enron, UCI, Yelp, ML-10M) and Temporal Knowledge Graphs (WIKI, YAGO).

Performance on Homogeneous and Bipartite Networks

NLSM consistently outperformed existing state-of-the-art (SOTA) models in link forecasting. On homogeneous networks, NLSM showed an average 4% improvement in Micro-AUC over models like DySAT. On bipartite networks, such as MovieLens (ML-10M), the improvement reached 8.7%.

Performance comparison on homogeneous and bipartite networks

Performance on Temporal Knowledge Graphs

On the WIKI and YAGO temporal knowledge graphs, NLSM significantly outperformed the previous leader, RE-NET. For the "WIKI-filtered" dataset, NLSM achieved a Mean Reciprocal Rank (MRR) of 56.70%, compared to RE-NET’s 53.57%.

Performance comparison on Temporal Knowledge Graphs (WIKI and YAGO)

Discussion and Future Work

The success of NLSM stems from the GRU-VI hybrid, which learns temporal patterns directly from data. Furthermore, the explicit interaction matrices $\Theta$ provide interpretability, allowing researchers to understand the drivers of link formation. Future work includes incorporating external node features and addressing memory bottlenecks for KGs with very large numbers of relations.

Summary updated

BibTeX
@inproceedings{gracious2021neural,
  author = {Tony Gracious and Shubham Gupta and Arun Kanthali and Rui M. Castro and Ambedkar Dukkipati},
  title = {{Neural Latent Space Model for Dynamic Networks and Temporal Knowledge Graphs}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i5.16526},
}
Figure