Learning Quantile Functions for Temporal Point Processes with Recurrent Neural Splines

Souhaib Ben TaiebAISTATS 2022 · OpenReview

Abstract

We propose a new neural parametrization for temporal point processes (TPP) based on the conditional quantile function. Specifically, we use a flexible monotonic rational-quadratic spline to learn a smooth continuous quantile function, with history conditioning achieved through a recurrent neural network. This novel parametrization, termed RQS-QF, provides a flexible yet tractable TPP model with advantages such as analytical sampling and closed-form expressions for quantiles and prediction intervals. We also derive a closed-form expression for the Continuous Ranked Probability Score (CRPS), enabling robust training. RQS-QF achieves state-of-the-art performance on both synthetic and real-world event data.

Introduction

Temporal Point Processes (TPPs) are essential for modeling sequences of events occurring at irregular intervals, with applications in finance, medicine, and social media. Traditional neural TPPs typically model the intensity function or the probability density function (PDF) of inter-arrival times. However, these approaches often struggle with efficient sampling and the calculation of prediction intervals, frequently requiring slow numerical integration. Furthermore, training via Maximum Likelihood Estimation (MLE) can be brittle and sensitive to outliers, as it only considers the probability at the exact moment an event occurs.

Methodology: Quantile Function Parametrization

The core innovation of this work is the direct modeling of the conditional quantile function $Q(\alpha)$ using recurrent rational-quadratic splines (RQS). Unlike intensity-based models, RQS-QF treats uncertainty as a primary output.

The RQS-QF model approximates the true quantile function of inter-arrival times using a monotonic rational-quadratic spline defined by a set of knots.

The model uses a Recurrent Neural Network (RNN) to process historical event data, which then outputs the parameters for a monotonic, continuously differentiable spline. This spline represents the quantile function for the next event time. A key mathematical contribution is the derivation of a closed-form expression for the Continuous Ranked Probability Score (CRPS). CRPS is a distance-sensitive loss function that is more robust than MLE, as it rewards the model for placing probability mass near the actual event time rather than just at the exact point.

Experimental Results

The RQS-QF model was evaluated on 17 different datasets, including 5 synthetic benchmarks and 12 real-world datasets covering social media interactions (Reddit, Twitter), consumer behavior (Yelp, LastFM), and technical logs (Wikipedia).

Comparison of RQS-QF with baseline models across various synthetic point processes, demonstrating superior performance in quantile and interval scores.

The model consistently achieved state-of-the-art performance across several metrics:

  • Accuracy: RQS-QF outperformed established baselines like LogNormMix and RMTPP in terms of quantile scores and interval scores across almost all real-world datasets.
  • Calibration: The model demonstrated excellent uncertainty calibration, with prediction intervals accurately reflecting the true distribution of events, measured by a very low Mean Absolute Calibration Error (MACE).
  • Efficiency: The closed-form CRPS allows for faster training and sampling compared to models relying on numerical approximations, especially when high precision is required.

Conclusion

RQS-QF introduces a flexible and mathematically tractable framework for TPPs by prioritizing quantile functions. By enabling analytical sampling and robust training through a closed-form CRPS, it addresses major limitations of existing density-based neural TPPs. While currently focused on univariate sequences, future work involves scaling this approach to multivariate processes where multiple interacting event types occur.

Paper Citation: Ben Taieb, S. (2022). Learning Quantile Functions for Temporal Point Processes with Recurrent Neural Splines. Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 151.

Summary updated

Abstract

We can build flexible predictive models for rich continuous-time event data by combining the framework of temporal point processes (TPP) with (recurrent) neural networks. We propose a new neural parametrization for TPPs based on the conditional quantile function. Specifically, we use a flexible monotonic rational-quadratic spline to learn a smooth continuous quantile function. Conditioning on historical events is achieved through a recurrent neural network. This novel parametrization provides a flexible yet tractable TPP model with multiple advantages, such as analytical sampling and closed-form expressions for quantiles and prediction intervals. While neural TPP models are often trained using maximum likelihood estimation, we consider the more robust continuous ranked probability score (CRPS). We additionally derive a closed-form expression for the CRPS of our model. Finally, we demonstrate that the proposed model achieves state-of-the-art performance in standard prediction tasks on both synthetic and real-world event data.
BibTeX
@inproceedings{taieb2022learning,
  author = {Souhaib Ben Taieb},
  title = {{Learning Quantile Functions for Temporal Point Processes with Recurrent Neural Splines}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2022},
  url = {https://openreview.net/forum?id=XJVHtNCn9y},
}
Figure