Abstract
The Neural Latent Space Model (NLSM) is a unified statistical framework designed to model the evolution of both simple (homogeneous) and complex (heterogeneous/Knowledge Graph) networks over time. By combining traditional latent space modeling with deep learning-based variational inference, NLSM achieves state-of-the-art performance in link forecasting while maintaining model interpretability.
Introduction
Most real-world networks—from social circles to international diplomacy—are dynamic. However, many representation learning techniques treat them as static snapshots. Existing methods for dynamic networks generally fall into two silos: those for homogeneous networks (one type of node and edge) and those for Knowledge Graphs (KGs) (multiple types of entities and relations). The challenge lies in creating a model that is unified, temporal, and scalable.
Methodology
The NLSM builds on several key pillars:
- Latent Space Representation: Each node is assigned a latent vector $z_n^{(t)}$ representing its attributes at time $t$.
- Interaction Matrices: The model uses interaction matrices $\Theta_k^{(t)}$ to define how specific attributes contribute to edge formation, capturing both homophily and heterophily.
- Minimalist Dynamics: Node attributes and interaction rules evolve via a Gaussian random walk, preventing overfitting to noise.
- Neural Variational Inference: The authors parameterize the variational distribution using Gated Recurrent Units (GRUs), which output the parameters for latent variables based on network history.
- Heterogeneous Extension: For KGs, the model assigns unique interaction matrices to each relation type, allowing the same node embeddings to interact differently depending on the relationship.
Experimental Evaluation
The model was evaluated across diverse dynamic networks, including homogeneous/bipartite datasets (Enron, UCI, Yelp, ML-10M) and Temporal Knowledge Graphs (WIKI, YAGO).
Performance on Homogeneous and Bipartite Networks
NLSM consistently outperformed existing state-of-the-art (SOTA) models in link forecasting. On homogeneous networks, NLSM showed an average 4% improvement in Micro-AUC over models like DySAT. On bipartite networks, such as MovieLens (ML-10M), the improvement reached 8.7%.
Performance on Temporal Knowledge Graphs
On the WIKI and YAGO temporal knowledge graphs, NLSM significantly outperformed the previous leader, RE-NET. For the "WIKI-filtered" dataset, NLSM achieved a Mean Reciprocal Rank (MRR) of 56.70%, compared to RE-NET’s 53.57%.
Discussion and Future Work
The success of NLSM stems from the GRU-VI hybrid, which learns temporal patterns directly from data. Furthermore, the explicit interaction matrices $\Theta$ provide interpretability, allowing researchers to understand the drivers of link formation. Future work includes incorporating external node features and addressing memory bottlenecks for KGs with very large numbers of relations.
Summary updated