Abstract
Relation discovery for multi-dimensional temporal point processes (MTPP) has received increasing interest for its importance in prediction and interpretability of the underlying dynamics. Traditional statistical MTPP models like Hawkes Process have difficulty in capturing complex relation due to their limited parametric form of the intensity function. While recent neural-network-based models suffer poor interpretability. In this paper, we propose a neural relation inference model namely TPP-NRI. Given MTPP data, it adopts a variational inference framework to model the posterior relation of MTPP data for probabilistic estimation. Specifically, assuming the prior of the relation is known, the conditional probability of the MTPP conditional on a sampled relation is captured by a message passing graph neural network (GNN) based MTPP model. A variational distribution is introduced to approximate the true posterior. Experiments on synthetic and real-world data show that our model outperforms baseline methods on both inference capability and scalability for high-dimensional data.
Introduction
Many real-world systems, from social media interactions to financial markets, can be modeled as Multi-dimensional Temporal Point Processes (MTPP). A critical challenge in analyzing these systems is relation inference: uncovering the underlying "who-influences-whom" network from observed event sequences.
Traditional statistical models, such as Hawkes Processes, often rely on rigid parametric forms that struggle to capture complex, non-linear dynamics. Conversely, modern neural models excel at prediction but often function as "black boxes." While attention mechanisms are sometimes used to infer relationships, their reliability as indicators of true physical influence is debated. Furthermore, existing relation inference frameworks like NRI are designed for regularly spaced time-series data, making them ill-suited for the irregular, bursty nature of MTPP events.
Methodology
The proposed TPP-NRI framework treats the relationship graph as a latent variable and employs a Variational Inference approach to learn a probability distribution over possible network structures. This allows for the incorporation of structural priors, such as sparsity.
The core of the model is a specialized Message Passing GNN Decoder that updates the system state through three distinct mechanisms:
- External Drive: Captures smooth state changes over time between events using exponential decay.
- Self-Update: Uses a Gated Recurrent Unit (GRU) to update a node's internal state when it fires an event.
- Affected-Update: Implements message passing; when an event occurs at node $i$, it sends a "message" to all influenced nodes $j$, modulating their future event intensities.
To ensure the discrete graph structure is end-to-end trainable, the model utilizes the Gumbel-Softmax trick for differentiable edge sampling.
Experiments and Results
The model was rigorously tested on synthetic Hawkes processes and real-world datasets including Stack Overflow interactions, email communications, and stock market fluctuations.
Relation Inference Performance
On synthetic data where the ground truth is known, TPP-NRI demonstrated superior capability in recovering the underlying graph. For instance, on Exponential Hawkes data, it achieved an F1-score of 0.954, significantly outperforming the traditional ADM4 baseline (0.897).
Scalability and Interpretability
TPP-NRI exhibits excellent scalability. In high-dimensional settings ($D=500$), it was approximately 10 times faster than ADM4 while maintaining higher accuracy. Unlike attention-based models, TPP-NRI provides a clear probabilistic interpretation: an edge weight of 0.8 directly corresponds to an 80% probability of influence. In the Stock dataset, the model successfully identified the strong relationship between Chevron and Exxon Mobil, the only two petroleum companies in the sample.
Conclusion
TPP-NRI provides a robust, interpretable, and scalable framework for discovering hidden relationships in complex event sequences. By integrating variational inference with message-passing GNNs, it bridges the gap between rigid statistical models and opaque neural predictors. Future work may explore dynamic relations where the influence graph evolves over time.
Summary updated