Abstract
Temporal point processes (TPPs) are stochastic process models used to characterize event sequences occurring in continuous time. Traditional statistical TPPs have a long-standing history, with numerous models proposed and successfully applied across diverse domains. In recent years, advances in deep learning have spurred the development of neural TPPs, enabling greater flexibility and expressiveness in capturing complex temporal dynamics. The emergence of large language models (LLMs) has further sparked excitement, offering new possibilities for modeling and analyzing event sequences by leveraging their rich contextual understanding. This survey presents a comprehensive review of recent research on TPPs from three perspectives: Bayesian, deep learning, and LLM approaches. We begin with a review of the fundamental concepts of TPPs, followed by an in-depth discussion of model design and parameter estimation techniques in these three frameworks. We also revisit classic application areas of TPPs to highlight their practical relevance. Finally, we outline challenges and promising directions for future research.
Introduction
Temporal point processes (TPPs) are the mathematical backbone for analyzing event sequences in continuous time. From financial transactions to social media interactions, TPPs model the timing and nature of discrete events. This survey maps the evolution of the field, categorizing research into three primary pillars: Bayesian, Neural, and LLM-based approaches.
The Core Challenge: Flexibility vs. Tractability
Many real-world phenomena generate event sequences at irregular intervals. Standard time-series models often struggle here because they assume fixed time steps. TPPs solve this by modeling the "intensity" or probability of an event happening at any given moment. However, the field has historically faced a trade-off: traditional models were often too rigid (parametric), while early nonparametric Bayesian models were mathematically elegant but computationally expensive. Modern research aims to break these barriers using deep learning and Large Language Models (LLMs).
Methodological Pillars
1. Bayesian Nonparametric TPPs
These models utilize Gaussian Processes (GPs) or Dirichlet processes as priors for the intensity function. This allows the model to "learn" the shape of the process from data without being forced into a predefined mathematical box (like a Poisson or Hawkes process). While flexible, these methods often remain computationally intensive.
2. Neural TPPs
Neural TPPs represent a significant shift toward high-capacity modeling. The survey categorizes these into three main types:
- Recurrent Models (RNN/LSTM): Efficient for streaming data but struggle with long-range dependencies.
- Autoregressive Models (Transformers): The current state-of-the-art for many tasks. They use self-attention to capture complex patterns across long histories and allow for parallel training.
- Differential Equation-based (ODE/SDE): These model the "hidden state" as a continuous flow, allowing the model to naturally handle the gaps between events.
3. LLM-based TPPs
The newest frontier involves integrating LLMs. Researchers are either using LLMs to "reason" about event causes (abductive reasoning) or encoding timestamps directly into tokens. This allows models to leverage the rich contextual understanding of LLMs, which is particularly effective in low-data regimes where traditional numerical models struggle.
Datasets and Benchmarks
While classic datasets in finance and social media remain relevant, new benchmarks are pushing the boundaries of the field:
- TESRBench: Focused on temporal event sequence retrieval from textual descriptions.
- DanmakuTPPBench: A multimodal benchmark involving video frames and time-stamped user comments, integrating TPPs with computer vision and NLP.
Key Insights and Innovations
- The "Intensity-Free" Revolution: A major breakthrough is moving away from modeling the intensity function $\lambda^*(t)$ and instead modeling the probability density $f(t)$ or the cumulative intensity $\Lambda^*(t)$ directly. This eliminates the need for complex numerical integration during training, making models faster and more stable.
- LLMs as Semantic Bridges: Traditional TPPs only care about time and category. LLM-based TPPs can ingest the actual text associated with an event (e.g., news alerts), allowing the model to understand why a surge is happening.
- Continuous-Time Latent States: By using Neural ODEs, models can represent how a system's internal state evolves smoothly even when no events are occurring, which is more physically realistic for many natural processes.
Results and Future Directions
Transformer-based TPPs have largely superseded RNNs for large-scale offline tasks due to their $O(N^2)$ training efficiency and ability to capture long-range triggers. Furthermore, LLM-based approaches (like LAMP) show significant improvements in event prediction by using "few-shot" reasoning.
However, challenges remain:
- Scalability: The quadratic complexity of Transformers remains a bottleneck for extremely long sequences.
- Sampling Efficiency: Generating new events remains slow due to reliance on sequential thinning algorithms.
- Interpretability: Neural models often act as "black boxes," hindering their adoption in critical fields like healthcare.
- Standardization: Establishing unified experimental setups to fairly compare different neural architectures.
Summary updated